Co-manipulation surgical system having a coupling mechanism removeably attachable to surgical instruments

ABSTRACT

Co-manipulation robotic systems are described herein that may be used for assisting with laparoscopic surgical procedures. The co-manipulation robotic systems allow a surgeon to use commercially-available surgical tools while providing benefits associated with surgical robotics. Advantageously, the surgical tools may be seamlessly coupled to the robot arms using a disposable coupler while the reusable portions of the robot arm remain in a sterile drape. Further, the co-manipulation robotic system may operate in multiple modes to enhance usability and safety, while allowing the surgeon to position the instrument directly with the instrument handle and further maintain the desired position of the instrument using the robot arm.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part application of U.S. Pat.Appl. No. 17/815,885, filed Jul. 28, 2022, now U.S. Pat. No. 11,504,197,which is a continuation application of PCT Patent Appl. No.PCT/IB2022/052989, filed Mar. 30, 2022, which claims priority to EPPatent Appl. No. 21306904.0, filed Dec. 22, 2021, EP Patent Appl. No.21306905.7, filed Dec. 22, 2021, EP Patent Appl. No. 21305929.8, filedJul. 5, 2021, and EP Patent Appl. No. 21305417.4, filed Mar. 31, 2021,the entire contents of each of which are incorporated herein byreference. This application claims the benefit of priority of U.S.Provisional Pat. Appl. No. 63/378,434, filed Oct. 5, 2022, and EP PatentAppl. No. 22306496.5, filed Oct. 5, 2022, the entire contents of each ofwhich are incorporated herein by reference. This application is relatedto U.S. Pat. Appl. No. 17/816,958, filed Aug. 2, 2022, the entirecontents of which are incorporated herein by reference.

FIELD OF USE

The present disclosure is directed to co-manipulation robotic systems,such as those having a coupling mechanism for removeably attaching asurgical instrument.

BACKGROUND

Managing vision and access during a laparoscopic procedure is achallenge. The surgical assistant paradigm is inherently imperfect, asthe assistant is being asked to anticipate and see with the surgeon’seyes, without standing where the surgeon stands, and similarly toanticipate and adjust how the surgeon wants the tissue of interestexposed, throughout the procedure. For example, during a laparoscopicprocedure, one assistant may be required to hold a retractor device toexpose tissue for the surgeon, while another assistant may be requiredto hold a laparoscope device to provide a field of view of the surgicalspace within the patient to the surgeon during the procedure, either oneof which may be required to hold the respective tools in an impracticalposition, e.g., from between the arms of the surgeon while the surgeonis actively operating additional surgical instruments.

Various attempts have been made at solving this issue. For example, arail-mounted orthopedic retractor, which is a purely mechanical devicethat is mounted to the patient bed/table, may be used to hold alaparoscope device in position during a laparoscopic procedure, andanother rail-mounted orthopedic retractor may be used to hold aretractor device in position during the laparoscopic procedure. However,the rail-mounted orthopedic retractor requires extensive manualinteraction to unlock, reposition, and lock the tool in position.

Complex robot-assisted systems such as the Da Vinci Surgical System(made available by Intuitive Surgical, Sunnyvale, California) have beenused by surgeons to enhance laparoscopic surgical procedures bypermitting the surgeon to tele-operatively perform the procedure from asurgeon console remote from the patient console holding the surgicalinstruments. Such complex robot-assisted systems are very expensive, andhave a very large footprint and take up a lot of space in the operatingroom. Moreover, such robot-assisted systems typically require uniquesystem-specific surgical instruments that are compatible with thesystem, and thus surgeons may not use standard off-the-shelf surgicalinstruments that they are used to. As such, the surgeon is required tolearn an entirely different way of performing the laparoscopicprocedure.

In view of the foregoing drawbacks of previously known systems andmethods, there exists a need for a system that provides the surgeon withthe ability to seamlessly position and manipulate various surgicalinstruments as needed, thus avoiding the workflow limitations inherentto both human and mechanical solutions.

SUMMARY

The present disclosure overcomes the drawbacks of previously-knownsystems and methods by providing a co-manipulation surgical system toassist with laparoscopic surgery performed using a surgical instrumenthaving a handle, an operating end, and an elongated shaft therebetween.The co-manipulation surgical system may include a robot arm having aproximal end, a distal end that may be removably coupled to the surgicalinstrument, a plurality of links, and a plurality of joints between theproximal end and the distal end. The co-manipulation surgical systemfurther may include a controller operatively coupled the robot arm. Thecontroller may be programmed to cause the robot arm to automaticallyswitch between: a passive mode responsive to determining that movementof the robot arm due to movement at the handle of the surgicalinstrument is less than a predetermined amount for at least apredetermined dwell time period, wherein the controller may beprogrammed to cause the robot arm to maintain a static position in thepassive mode; and a co-manipulation mode responsive to determining thatforce applied at the robot arm due to force applied at the handle of thesurgical instrument exceeds a predetermined threshold, wherein thecontroller may be programmed to permit the robot arm to be freelymoveable in the co-manipulation mode responsive to movement at thehandle of the surgical instrument for performing laparoscopic surgeryusing the surgical instrument, and wherein the controller may beprogrammed to apply a first impedance to the robot arm in theco-manipulation mode to account for weight of the surgical instrumentand the robot arm. The controller further may be programmed to cause therobot arm to automatically switch to a haptic mode responsive todetermining that at least a portion of the robot arm is outside apredefined haptic barrier, wherein the controller may be programmed toapply a second impedance to the robot arm in the haptic mode greaterthan the first impedance, thereby making movement of the robot armresponsive to movement at the handle of the surgical instrument moreviscous in the haptic mode than in the co-manipulation mode

In addition, the co-manipulation surgical system may include a baserotatably coupled to the proximal end of the robot arm, such that therobot arm may move relative to the base. For example, the base may berotatable about a first axis, such that rotation of the base causesrotation of the robot arm about the first axis. Accordingly, the systemfurther may include a first motor disposed within the base andoperatively coupled to the base, such that the controller is operativelycoupled to the first motor and programmed to cause the first motor toapply impedance to the base. Moreover, a proximal end of a shoulder linkof the plurality of links may be rotatably coupled to the base at ashoulder joint of the plurality of joints, such that rotation of theshoulder link causes rotation of links of the plurality of links distalto the shoulder link about a second axis of the shoulder joint.Accordingly, the system further may include a second motor disposedwithin the base and operatively coupled to the shoulder joint, such thatthe controller is operatively coupled to the second motor and programmedto cause the second motor to apply impedance to the shoulder joint. Forexample, the second axis may be perpendicular to the first axis.

Further, a proximal end of an elbow link of the plurality of links mayrotatably coupled to a distal end of the shoulder link at an elbow jointof the plurality of joints, such that rotation of the elbow link causesrotation of links of the plurality of links distal to the elbow linkabout a third axis of the elbow joint. Accordingly, the system furthermay include a third motor disposed within the base and operativelycoupled to the elbow joint, such that the controller is operativelycoupled to the third motor and programmed to cause the third motor toapply impedance to the elbow joint. The shoulder link may include aproximal shoulder link rotatably coupled to the base and a distalshoulder link rotatably coupled to the elbow link. The distal shoulderlink may be rotatable relative to the proximal shoulder link, such thatrotation of the distal shoulder link relative to the proximal shoulderlink causes rotation of links of the plurality of links distal to thedistal shoulder link to rotate about a fourth axis parallel to alongitudinal axis of the shoulder link.

The system further may include an actuator that may be actuated topermit rotation of the distal shoulder link relative to the proximalshoulder link, wherein, in an unactuated state, the actuator preventsrotation of the distal shoulder link relative to the proximal shoulderlink. In addition, a proximal end of a wrist link of the plurality oflinks may be rotatably coupled to a distal end of the elbow link at aproximal wrist joint of the plurality of joints, such that the wristlink may be rotated relative to the elbow link about a fifth axis of theproximal wrist joint. The system further may include an actuator thatmay be actuated to permit rotation of the wrist link relative to theelbow link, wherein, in an unactuated state, the actuator preventsrotation of the wrist link relative to the elbow link. The wrist linkmay include a proximal wrist link rotatably coupled to the distal end ofthe elbow link, a middle wrist link rotatably coupled to proximal wristlink about a sixth axis, and a distal wrist link rotatably coupled tothe middle wrist link about a seventh axis. The distal wrist link may beremovably coupled to the surgical instrument.

The system further may include a platform coupled to the base. Theplatform may permit vertical and horizontal movement of the baserelative to the platform, to thereby cause vertical and horizontalmovement of the robot arm relative to the platform. The platform mayinclude a plurality of wheels that may permit mobility of the platform,the plurality wheels having a brake mechanism that may be actuated toprevent mobility of the platform. Moreover, the controller may beprogrammed to receive information associated with the surgicalinstrument coupled to the distal end of the robot arm, the informationincluding at least one of instrument type, weight, center of mass,length, or instrument shaft diameter.

The system further may include a database having information associatedwith a plurality of surgical instruments, wherein the controller isprogrammed to access the database to retrieve the information associatedwith the surgical instrument coupled to the distal end of the robot arm.In addition, the system may include an optical scanner that may measuredepth data, such that the controller is programmed to identify thesurgical instrument coupled to the distal end of the robot arm based onthe measured depth data. Moreover, the controller may be programmed tobe calibrated to the surgical instrument when the surgical instrument iscoupled to the distal end of the robot arm.

The system further may include a base housing at the proximal end of therobot arm, and motors for controlling the robot arm, such that all themotors for the robot arm are disposed within the base housing. Forexample, the system further may include a base rotatably coupled to theproximal end of the robot arm, such that the robot arm may move relativeto the base, and a plurality of motors disposed within the base that areoperatively coupled to at least some joints of the plurality of joints,such that wherein the controller is operatively coupled to the pluralityof motors and programmed to measure current of the plurality of motors.

The controller further may be programmed to calculate a force applied tothe distal end of the robot arm based on the measured current of theplurality of motors. Moreover, the controller may be programmed todetermine a point of entry of the surgical instrument into a patient inreal-time based on a longitudinal axis of the surgical instrument whenthe surgical instrument is coupled to the distal end of the robot arm.For example, the controller may be programmed to determine the point ofentry of the surgical instrument into the patient in real-time bydetermining a point of intersection of a plurality of virtual linesparallel to the longitudinal axis of the surgical instrument as thesurgical instrument moves relative to the point of entry. In addition,the controller may be programmed to calculate a force applied to theoperating end of the surgical instrument based on the force applied tothe distal end of the robot arm, the length of the surgical instrument,the center of mass of the surgical instrument, and the point of entry.Additionally, the controller may be programmed to calculate a forceapplied to the patient at the point of entry of the surgical instrumentinto the patient based on the force applied to the distal end of therobot arm, the center of mass of the surgical instrument, and the pointof entry. The controller further may be programmed to detect a faultcondition of the co-manipulation surgical system, and wherein, if amajor fault condition is detected, the controller may cause actuation ofbrakes of the plurality of motors. Moreover, the controller may beprogrammed to apply a third impedance to the robot arm to resistmovement of the robot arm if the force applied to the distal end of therobot arm exceeds a predetermined force threshold within a predeterminedtime period.

The system further may include a plurality of encoders disposed on atleast some joints of the plurality of joints, wherein the plurality ofencoders may measure angulation of corresponding links of the pluralityof links at the at least some joints, such that the controller may beprogrammed to determine a position of the distal end of the robot arm in3D space based on the angulation measurements by the plurality ofencoders. In addition, the system may include one or more indicatorsdisposed on at least one link of the plurality of links of the robotarm, wherein the one or more indictors may illuminate a plurality ofcolors, each color indicative of a state of the co-manipulation surgicalsystem. For example, a first color of the plurality of colors mayindicate that the robot arm is in the passive mode, a second color ofthe plurality of colors may indicate that the robot arm is in theco-manipulation mode, and a third color of the plurality of colors mayindicate that the robot arm is in the haptic mode. Moreover, a fourthcolor of the plurality of colors may indicate a fault condition of theco-manipulation surgical system is detected by the controller.Additionally, a fifth color of the plurality of colors may indicate thatno surgical instrument is coupled to the distal end of the robot arm.

The predefined haptic barrier may be used to guide the surgicalinstrument coupled to the distal end of the robot arm to assist with thelaparoscopic surgery. For example, the predefined haptic barrier may bea haptic funnel that may guide the surgical instrument coupled to thedistal end of the robot arm into a trocar. The controller may beprogrammed to apply a third impedance to the robot arm to account forweight of the robot arm when no surgical instrument is coupled to thedistal end of the robot arm. Moreover, in the passive mode, thecontroller may be programmed to apply a third impedance to the robot armto account for weight of the surgical instrument, the weight of therobot arm, and a force applied to the distal end of the robot arm due toan external form applied to the surgical instrument to cause the robotarm to maintain the static position.

The system further may include a graphical user interface that maydisplay information associated with the surgical instrument coupled tothe distal end of the robot arm. The graphical user interface may permita user to adjust at least one of: the predetermined amount of movementat the handle of the surgical instrument or the predetermined dwell timeperiod to cause the robot arm to automatically switch to the passivemode, the predetermined threshold of force applied at the handle of thesurgical instrument to cause the robot arm to automatically switch tothe co-manipulation mode, a position of the predefined haptic barrier,an identity of the surgical instrument coupled to the distal end of therobot arm, a vertical height of the robot arm, or a horizontal positionof the robot arm.

The system further may include a coupler body that may be removablycoupled to a coupler interface disposed at the distal end of the robotarm. The coupler body may have a lumen sized and shaped to receive theelongated shaft of the surgical instrument therethrough, may transitionbetween an open state where the elongated shaft is slidably moveablewithin the lumen, and a closed state where longitudinal movement of theelongated shaft relative to the coupler body is inhibited whilerotational movement of the elongated shaft relative to the coupler bodyis permitted responsive to movement at the handle of the surgicalinstrument. For example, when the coupler body is coupled to the couplerinterface in the closed state, the robot arm may be permitted to befreely moveable responsive to movement at the handle of the surgicalinstrument for performing laparoscopic surgery if the force applied atthe robot arm due to force applied at the handle of the surgicalinstrument exceeds the predetermined threshold. In the closed state,longitudinal movement of the elongated shaft relative to the couplerbody may be inhibited while rotational movement of the elongated shaftrelative to the coupler body is permitted responsive to movement at thehandle of the surgical instrument due to frictional forces between thelumen of the coupler body and the elongated shaft of the surgicalinstrument.

In addition, the coupler body may be removably coupled to the couplerinterface via a magnetic connection. The controller may be programmed todetermine an orientation of the surgical instrument relative to thedistal end of the robot arm when the coupler body is coupled to thecoupler interface based on an alignment of the magnetic connection. Thesystem further may include a sterile drape that may be disposed betweenthe coupler body and the coupler interface, such that the sterile drapeprevents contact between the surgical instrument and the robot armduring the laparoscopic surgery. The distal end of the robot arm may beremovably coupled to at least one of a laparoscope, a retractor tool, agrasper tool, or a surgical cutting tool. For example, when the distalend of the robot arm is coupled to a laparoscope, the controller may beprogrammed to optically track an end-effector of one or more surgicalinstruments within a field of view of the laparoscope, and to cause therobot arm to automatically switch to a robotic assist mode responsive todetermining that the end-effector of the one or more surgicalinstruments are not within a predefined boundary within the field ofview of the laparoscope. Moreover, the controller may be programmed tocause the robot arm to move the laparoscope to adjust the field of viewof the laparoscope such that the end-effector of the one or moresurgical instruments are within the predefined boundary within the fieldof view of the laparoscope.

The co-manipulation surgical system may not be teleoperated via userinput received at a remote surgeon console. In addition, theco-manipulation surgical system may be structured such that a surgeonperforming the laparoscopic surgery does not contact any portion of theco-manipulation surgical system to move the surgical instrument whileperforming the laparoscopic surgery. Moreover, the system may include anoptical scanner, e.g., a LiDAR device, for measuring depth data. Forexample, the controller may be programmed to determine whether amovement applied to the surgical instrument coupled to the distal end ofthe robot arm is by an intended user. Additionally, the controller maybe programmed to identify the surgical instrument coupled to the distalend of the robot arm based on the depth data.

In addition, the system may include a second robot arm having a proximalend, a distal end that may be removably coupled to a second surgicalinstrument having a handle, an operating end, and an elongated shafttherebetween, a plurality of links, and a plurality of joints betweenthe proximal end and the distal end. Accordingly, the controller may beoperatively coupled the second robot arm, and programmed to cause thesecond robot arm to automatically switch between: the passive moderesponsive to determining that movement of the second robot arm due tomovement at the handle of the second surgical instrument is less than apredetermined amount for at least a predetermined dwell time periodassociated with the second robot arm, wherein the controller may beprogrammed to cause the second robot arm to maintain a static positionin the passive mode; the co-manipulation mode responsive to determiningthat force applied at the second robot arm due to force applied at thehandle of the second surgical instrument exceeds a predeterminedthreshold associated with the second robot arm, wherein the controllermay be programmed to permit the second robot arm to be freely moveablein the co-manipulation mode responsive to movement at the handle of thesecond surgical instrument for performing laparoscopic surgery using thesecond surgical instrument, and wherein the controller may be programmedto apply a third impedance to the second robot arm in theco-manipulation mode to account for weight of the second surgicalinstrument and the robot arm; and optionally the haptic mode responsiveto determining that at least a portion of the second robot arm isoutside the predefined haptic barrier, the controller may be programmedto apply a fourth impedance to the second robot arm in the haptic modegreater than the third impedance, thereby making movement of the secondrobot arm responsive to movement at the handle of the second surgicalinstrument more viscous in the haptic mode than in the co-manipulationmode.

In accordance with another aspect of the present disclosure, aco-manipulation robotic surgical device for manipulating an instrumentis provided. The device may include a base portion, a first arm coupledwith the base portion, a motor coupled with the first arm that mayrotate the first arm relative to the base portion, an instrument coupledwith an end portion of the first arm, and a controller that may beprogrammed to control the first arm according to at least two of thefollowing operational modes: passive assistant mode; co-manipulationassistant mode; robotic assistant mode; and haptic mode. For example, inthe passive assistant mode, the first arm is static. In theco-manipulation assistant mode, the first arm may be freely movable byan operator while the motor at least partially simultaneously moves thefirst arm to improve a position and/or orientation of the instrumentcoupled with the end portion of the first arm and/or to compensate atleast for a force of gravity on the first arm and the instrument that iscoupled with the end portion of the first arm. In the robotic assistantmode, the motor may move the first arm to reposition the instrumentcoupled with the end portion of the first arm. In the haptic mode, thefirst arm may be movable by an operator while the motor compensates atleast for a force of gravity on the first arm and/or the instrument thatis coupled with the end portion of the first arm and at least guides theinstrument along a predefined trajectory, prevents unwanted movements ofthe first arm and/or the instrument coupled with the end portion of thefirst arm, prevents a movement of the first arm outside of a particularspace, and/or prevents a movement of the first arm into a particularspace.

In one embodiment, the controller may be switchable between any one ofat least three of the operational modes. Alternatively, the controllermay be switchable between any one of the four operational modes. Theco-manipulation robotic surgical device may be programmed toautomatically identify the particular instrument that is coupled withthe end portion of the first arm using an RFID transmitter chip, abarcode, a near field communication device, a Bluetooth transmitter,and/or a weight of the instrument that is coupled with the end portionof the first arm. Moreover, the co-manipulation robotic surgical devicemay be programmed to automatically change to a predetermined one of theoperational modes when a particular instrument is coupled with the endportion of the first arm without any additional input from an operator.For example, the co-manipulation robotic surgical device may beprogrammed to change to the passive assistant mode when a particularinstrument is coupled with the end portion of the first arm without anyadditional input from an operator.

In accordance with another aspect of the present invention, anotherco-manipulation surgical system to assist with laparoscopic surgeryperformed using a surgical instrument having a handle, an operating end,and an elongated shaft therebetween is provided. The co-manipulationsurgical system may include a robot arm having a proximal end, a distalend that may be removably coupled to the surgical instrument, aplurality of links, and a plurality of joints between the proximal endand the distal end. The distal end of the robot arm may include acoupler interface. The system further may include a coupler body thatmay be removably coupled to the coupler interface. The coupler body mayinclude a lumen sized and shaped to receive the elongated shaft of thesurgical instrument therethrough, and may to transition between an openstate where the elongated shaft is slidably moveable within the lumen,and a closed state where longitudinal movement of the elongated shaftrelative to the coupler body is inhibited while rotational movement ofthe elongated shaft relative to the coupler body is permitted responsiveto movement at the handle of the surgical instrument. For example, whenthe coupler body is coupled to the coupler interface in the closedstate, the robot arm is permitted to be freely moveable responsive tomovement at the handle of the surgical instrument for performinglaparoscopic surgery.

The coupler body may be removably coupled to the coupler interface via amagnetic connection. Accordingly, the controller may be programmed todetermine an orientation of the surgical instrument relative to thedistal end of the robot arm when the coupler body is coupled to thecoupler interface based on an alignment of the magnetic connection. Thesystem further may include a sterile drape that may be disposed betweenthe coupler body and the coupler interface, such that the sterile drapeprevents contact between the surgical instrument and the robot armduring the laparoscopic surgery. The coupler body may be disposableafter a single laparoscopic surgery.

In accordance with another aspect of the present invention, a device forcoupling an instrument, e.g., a laparoscopic surgical instrument or anendoscope, to an arm of a surgical robot is provided. The device mayinclude a body sized and shaped to selectively couple with an instrumentfor use in a surgical operation, and an interface that may selectivelycouple with the body and may be coupled with an end portion of a roboticarm. For example, the device may permit the instrument to rotate about alongitudinal axis of the instrument relative to the device, and furthermay inhibit longitudinal movement of the instrument relative to thedevice. The body may clamp around a portion of an outside surface of theinstrument. For example, the body may include a first portion coupledwith a second portion with a hinge, wherein the first portion may rotateabout the hinge relative to the second portion so as to selectivelyclamp the instrument in a recess formed in the body.

In addition, the body may clamp around a portion of an outside surfaceof the instrument and prevent a rotational movement of the instrumentrelative to the body under normal operating conditions. For example, theinterface may include a recess sized and shaped to removably receive thebody therein. The recess of the interface may inhibit longitudinalmovement of the body relative to the interface and permit rotationalmovement of the body relative to the interface. Moreover, the device maymove between a first state in which the instrument is removable from thedevice and a second state in which the instrument is nonremovable fromthe device. The body may have one or more projections extending awayfrom a surface of the body and the interface may have one or moredepressions for receiving the one or more projections to align the bodywith the interface.

In accordance with yet another aspect of the present invention, aco-manipulation surgical robot system for performing a surgicalprocedure is provided. The system may include a first surgical robothaving a base, an arm coupled with the base, and a motor coupled withthe arm and that may move the arm relative to the base, as well as acontroller programmed to control the arm, and an optical scanner thatmay collect depth data. For example, the optical scanner may collectdepth data related to a position and an orientation of an instrumentwith respect to the co-manipulation surgical robot. The system may beprogrammed to use the depth data to determine if the instrument iscoupled with the first surgical robot. Moreover, the system may beprogrammed to determine an identity of the instrument based at least inpart on the depth data.

The optical scanner may collect depth data related to a position and amovement of an instrument, wherein the instrument may be freely held bya surgeon and not coupled with a surgical robot. Moreover, the opticalscanner may collect depth data related to a trocar inserted into thepatient. Accordingly, the system may be programmed to move the armand/or the base of the first surgical robot if the position of thetrocar changes more than a threshold amount. The system further mayinclude a second surgical robot having a second base, a second armcoupled with the second base, a second motor coupled with the second armand that may move the second arm relative to the second base. Theoptical scanner may have an accuracy of at least 5 mm at a range of 10meters. The optical scanner further may collect depth data related to asurgeon’s hand during a surgical procedure.

Moreover, the controller may be programmed to control the arm of thefirst surgical robot according to at least one of the followingoperational modes: passive assistant mode; co-manipulation assistantmode; robotic assistant mode; and haptic mode, as described above. Theoptical scanner may use the depth data to identify a potentialinadvertent collision between the arm of the first surgical robot and apatient, a support platform supporting at least the first surgicalrobot, another surgical robot, and/or another object in an operatingroom and to warn a user of the potential inadvertent collision and/orinhibit a movement of the arm of the first surgical robot and/or causemovement of the arm of the first surgical robot to avoid such acollision. In addition, the first surgical robot may be supported by asupport platform and wherein the co-manipulation surgical robot systemmay be programmed to move the first surgical robot relative to thesupport platform based on the depth data collected by the opticalscanner to optimize a position of the first surgical robot on thesupport platform. In addition, the optical scanner may collect depthdata used to record a movement of a surgeon’s hand during a surgicalprocedure.

In accordance with another aspect of the present invention, anotherco-manipulation surgical robot system for performing a surgicalprocedure is provided. The system may include a surgical robot having abase, an arm coupled with the base, and a motor coupled with the arm, aswell as an optical scanner that may track a movement of one or moreobjects around a patient, and a controller programmed to collect datafrom the optical sensor regarding the movement of one or more objectsand to move the arm of the surgical robot in response to the movement ofone or more objects.

In accordance with another aspect of the present invention, aco-manipulation robotic surgical system for assisting in themanipulation of an instrument is provided. The system may include abase, an arm coupled with the base, the arm having a plurality of armsegments and a plurality of articulation joints, a plurality of motorscoupled with the arm, wherein the plurality of motors may rotate theplurality of arm segments about the plurality of articulation joints,and a controller programmed to control at least the plurality of motors.For example, the arm may be movable by a user exerting a force directlyon the arm and/or directly on an instrument coupled with the arm.Moreover, the system may be programmed to collect data related to afirst operating characteristic of the arm and/or an instrument coupledwith the arm. Additionally, the controller may be programmed to analyzethe data related to the first operating characteristic to detect whethera first condition exists, and to modify a first operating parameter ofthe arm if the first condition is detected.

The system may be programmed to compare the data collected during asurgical procedure with historical data related to the same surgicalprocedure for a same user using the instrument to detect if the firstcondition exists. The system further may include an optical scanner, oneor more sensors positioned on the arm, and/or an endoscope to collectdata related to the first operating characteristic of the arm and/or aninstrument coupled with the arm. The controller may be programmed toautomatically change a position and/or an orientation of an imagingdevice supported by the arm to a preferred or optimal position and/ororientation if a position and/or an orientation of the imaging device isnot the preferred or the optimal position of the camera for capturing animage of the instrument. In addition, the controller may be programmedto detect if an instrument coupled with the arm is replaced.

In addition, the system may be programmed to detect a magnitude andduration of one or more forces applied to the first robotic arm, andfurther to detect that the first condition exists if a change in a forceapplied to the arm meets or exceeds a first predetermined value over athreshold duration of time. The system further may be programmed tocalculate an actual direction or an actual approximate direction that anend effector at a distal end of the arm is pointing to and a calculateddirection or a calculated approximate direction that the end effectorwould be pointing to if an instrument were coupled with the end effectorand to compare the actual direction or the actual approximate directionwith the calculated direction or the calculated approximate directionand determine if the actual direction or the actual approximatedirection and the calculated direction or the calculated approximatedirection are different. The controller may be programmed such that, ifa first instrument coupled with the arm is replaced by a secondinstrument, the controller updates a data file associated with thesecond instrument, wherein the data file associated with the secondinstrument includes at least a center of gravity of the secondinstrument and viscosity parameter of the second instrument.

In addition, the controller may be programmed to detect if a magnitudeof force exerted at a distal end of an instrument coupled with the armequals or exceeds a first value and/or if a magnitude of a force exertedon a trocar through which the instrument passes equals or exceeds asecond value and to provide an alert to a user of the arm if themagnitude of force exerted at the distal end of the instrument coupledwith the arm equals or exceeds the first value and/or if the magnitudeof the force exerted on the trocar through which the instrument passesequals or exceeds the second value. Moreover, the controller may beprogrammed to detect if a dwell time of the arm and/or an instrumentcoupled with the arm equals or exceeds a threshold dwell time, andfurther to change an operational state of the arm to a static hold stateif the dwell time of the arm and/or an instrument coupled with the armequals or exceeds the threshold dwell time, wherein the dwell time is anamount of time that the arm and/or an instrument coupled with the arm isheld in a static position.

In the static hold state, the system may be programmed to hold the armin a static position and to inhibit a movement of the arm from thestatic position of the arm except when a force applied to the arm and/oran instrument held by the arm by a user of the system equals or exceedsa predefined threshold release force value. The arm and/or an instrumentcoupled with the arm may be considered to be held in a static positionwhen the arm is not moved more than 5 mm in any direction during thedwell time. In some embodiments, the threshold dwell time may be lessthan one-half of a second. In addition, the controller may be programmedto detect whether a user is attempting to remove a first instrument fromthe arm, such that the controller may be programmed to reduce a couplingforce applied by the arm to the first instrument if the controllerdetects that the user is attempting to remove the first instrument fromthe arm.

The system further may include a support platform for supporting atleast the base. Accordingly, the controller may be programmed to detectwhether a surgical procedure is being initiated, and to move the supportplatform supporting the base to an initial position and/or the arm to aninitial position and/or orientation for the particular surgicalprocedure before the surgical procedure has started if the controllerdetects that a surgical procedure is being initiated.

In accordance with yet another aspect of the present invention, anotherco-manipulation robotic surgical system for assisting in themanipulation of an instrument is provided. The system may include abase, an arm coupled with the base, the arm having a plurality of armsegments and a plurality of articulation joints, a plurality of motorscoupled with the arm, wherein the plurality of motors may rotate theplurality of arm segments about the plurality of articulation joints,and a controller programmed to control at least the plurality of motors.For example, the arm may be movable by a user exerting a force directlyon the arm and/or directly on an instrument coupled with the arm. Uponan identification of a first user, the system may be programmed toautomatically load a data file associated with the first user comprisingat least a first operating parameter configured to modify an operatingcharacteristic of the co-manipulation robotic surgical system.Accordingly, the controller may be programmed to control the pluralityof motors according to at least the first operating parameter.

The first operating parameter of the data file associated with the firstsurgeon may be based at least in part on data collected during priorsurgical procedures performed by the first user. Additionally, the firstoperating parameter of the data file associated with the first user maybe based at least in part on manually entered preferences for the firstuser. The system may be programmed to automatically identify the firstuser using an optical scanner. In addition, the co-system may beprogrammed to automatically load the data file associated with the firstuser upon manual input of an identity of the first user. The data fileassociated with the first user may include a threshold dwell time valuebased on dwell time data collected from procedures performed by thefirst user and/or preferences manually input for the first user.Moreover, the data file associated with the first user may include adwell speed value based on data collected from procedures performed bythe first user and/or preferences manually input for the first user.

In addition, the data file associated with the first user may include alaparoscopic view parameter based on laparoscopic view data collectedfrom procedures performed by the first user, such that the controllermay be programmed to automatically change a position and/or anorientation of a laparoscope according to the laparoscopic view datacollected from procedures performed by the first user. The data fileassociated with the first user may include a setup joint parameter basedon setup joint position data collected from past procedures performed bythe first user. In addition, the data file may include instrumentcalibration parameters based on instrument calibration values input bythe first user. The first operating parameter may be based on at leastone of a pose of the first user, a height of the first user, or a handpreference of the first user.

Moreover, the controller may be programmed to automatically detect whenthe instrument coupled with the arm is not in an optimal or preferredlocation based on data collected from procedures performed by the firstuser and to move the arm so that the instrument is in the optimal orpreferred location. In addition, the system may be programmed to detectwhen the first user desires to change an operating mode of the system toa static hold mode even when a dwell time of the arm and/or aninstrument coupled with the arm is less than a threshold dwell time. Thedata file may be communicable from a network database in communicationwith the co-manipulation surgical robot system. Additionally, the firstoperating parameter of the data file associated with the first user maybe based at least in part on data collected during prior surgicalprocedures performed by a plurality of users.

In accordance with another aspect of the present disclosure, aco-manipulation surgical robot system for performing a surgicalprocedure is provided. The system may include a first surgical robothaving a base, an arm coupled with the base, and a motor coupled withthe arm and configured to move the arm relative to the base. The systemfurther may include a controller programmed to control the arm, and anoptical scanner that collects depth data. The optical scanner maycollect depth data related to a position and an orientation of aninstrument with respect to the co-manipulation surgical robot system.For example, the controller may be programmed to determine if theinstrument is coupled with the first surgical robot based on the depthdata. In addition, the controller may be programmed to identify a typeof the instrument based at least in part on the depth data.

Moreover, the optical scanner may collect depth data related to aposition and a movement of an instrument, wherein the instrument isfreely held by a surgeon and not coupled with a surgical robot. Inaddition, the optical scanner may collect depth data related to a trocarinserted into the patient. The system may be configured to move the armand/or the base of the first surgical robot if the position of thetrocar changes more than a threshold amount. Additionally, the systemfurther may include a second surgical robot having a second base, asecond arm coupled with the second base, and a second motor coupled withthe second arm and configured to move the second arm relative to thesecond base. The optical scanner may have an accuracy of at least 5 mmat a range of 10 meters. In addition, the optical scanner may collectdepth data related to a surgeon’s hand during a surgical procedure.

The controller may be programmed to control the arm of the firstsurgical robot according to at least one of the following operationalmodes: passive assistant mode; co-manipulation assistant mode; roboticassistant mode; and haptic mode. For example, in the passive assistantmode, the arm may be static. In the co-manipulation assistant mode, thearm may be freely movable by an operator while the motor at leastpartially simultaneously moves the arm to improve a position and/ororientation of the instrument coupled with an end portion of the armand/or to compensate at least for a force of gravity on the arm and theinstrument that is coupled with the end portion of the arm. In therobotic assistant mode, the motor may move the arm to reposition theinstrument coupled with the end portion of the arm. In the haptic mode,the arm may be movable by an operator while the motor compensates atleast for a force of gravity on the arm and/or the instrument that iscoupled with the end portion of the arm and at least guides theinstrument along a predefined trajectory, prevents unwanted movements ofthe arm and/or the instrument coupled with the end portion of the arm,prevents a movement of the arm outside of a particular space, and/orprevents a movement of the arm into a particular space.

The optical scanner may use the depth data to identify a potentialinadvertent collision between the arm of the first surgical robot and atleast one of a patient, a support platform supporting at least the firstsurgical robot, another surgical robot, and/or another object in anoperating room, to warn a user of the potential inadvertent collision,and/or to inhibit a movement of the arm of the first surgical robot toavoid such a collision. In addition, the first surgical robot may besupported by a support platform, such that the co-manipulation surgicalrobot system may be configured to move the first surgical robot relativeto the support platform based on the depth data collected by the opticalscanner to optimize a position of the first surgical robot on thesupport platform. The optical scanner may collect depth data used torecord a movement of a surgeon’s hand during a surgical procedure.

Moreover, the first surgical robot may be supported by a supportplatform having a plurality of wheels that permit mobility of thesupport platform. The plurality of wheels may include a brake mechanismconfigured to be engaged to prevent mobility of the support platform.The system further may include one or more additional optical scannersdisposed on the support platform and configured to collect depth data.For example, at least one of the optical scanner or the one or moreadditional optical scanners may include at least one of a depth camera,a stereo RGB camera, a LIDAR device, or an electromagnetic, capacitive,or infrared proximity sensor. In addition, the system may include adisplay operatively coupled to the controller. The controller may beprogrammed to generate a map of an area surrounding the support platformbased on the depth data collected from at least one of the opticalscanner or the one or more optical scanners, and to cause the display todisplay the map. The map generated by the controller may includegraphical representations of the support platform relative to one ormore objects and/or persons within the area surrounding the supportplatform.

In addition, the controller may be programmed to generate an alert ifthe map indicates that the support platform is within a predetermineddistance from the one or more objects and/or persons within the areasurrounding the support platform. The system further may include anactuator operatively coupled to the brake mechanism. The actuator may beactuated to disengage the brake mechanism to permit mobility of thesupport platform, such that the controller automatically causes thedisplay to display the map when the brake mechanism is disengaged.

In accordance with yet another aspect of the present disclosure, aco-manipulation surgical robot system for performing a surgicalprocedure is provided. The system may include a surgical robot having abase, an arm coupled with the base, and a motor coupled with the arm.The system further may include an optical scanner configured to track amovement of one or more objects around a patient, and a controllerprogrammed to collect data from the optical sensor regarding themovement of one or more objects and to move the arm of the surgicalrobot in response to the movement of one or more objects. In addition,the controller may be programmed to cause the base to move in at leastone degree of freedom.

In accordance with another aspect of the present disclosure, aco-manipulation surgical system to assist with laparoscopic surgeryperformed using a surgical instrument having a handle, an operating end,and an elongated shaft therebetween is provided. The system may includea robot arm, a platform configured to support the robot arm, a pluralityof optical sensors coupled to the platform and configured to collectdepth data, a display operatively coupled to the platform, and acontroller programmed to permit the robot arm to be freely moveableresponsive to movement at the handle of the surgical instrument forperforming laparoscopic surgery using the surgical instrument. The robotarm may include a proximal end, a distal end configured to be removablycoupled to the surgical instrument, a plurality of links, and aplurality of joints. The platform may include a plurality of wheels thatpermit mobility of the platform. In addition, the controller may beprogrammed to: receive the depth data collected by the plurality ofoptical sensors; generate a map of an area surrounding the platformbased on the depth data; and cause the display to display the map.

The plurality of optical sensors may be at least one of a depth camera,a stereo RGB camera, a LIDAR device, or an electromagnetic, capacitive,or infrared proximity sensor. The map generated by the controller mayinclude graphical representations of the platform relative to at leastone of one or more objects or one or more persons within the areasurrounding the platform. The controller further may generate an alertif the map indicates that the platform is within a predetermineddistance from the at least one of one or more objects or one or morepersons within the area surrounding the support platform. In addition,the system may include a brake mechanism configured to be engaged toprevent mobility of the support platform, and an actuator operativelycoupled to the brake mechanism. The actuator may be actuated todisengage the brake mechanism to permit mobility of the supportplatform, such that the controller may automatically cause the displayto display the map when the brake mechanism is disengaged. Moreover, thecontroller may be programmed to cause the robot arm to move in at leastone degree of freedom relative to the platform, e.g., the controller maycause vertical and horizontal movement of the robot arm relative to theplatform.

In accordance with another aspect of the present disclosure, a methodfor assisting with laparoscopic surgery using a robot arm having aplurality of links, a plurality of joints, a proximal end supported by aplatform having a plurality of wheels that permit mobility of theplatform, and a distal end configured to be removably coupled to asurgical instrument is provided. The method may include collecting depthdata from a plurality of optical scanners coupled to the platform;generating a map of an area surrounding the platform based on the depthdata, the map including graphical representations of the platformrelative to at least one of one or more objects or one or more personswithin the area surrounding the platform; and causing a display todisplay the map while the platform is moving to guide movement of theplatform within an operating room.

In accordance with yet another aspect of the present disclosure, aco-manipulation surgical system to assist with laparoscopic surgeryperformed using a surgical instrument having a handle, an operating end,and an elongated shaft therebetween is provided. The system may includea robot arm, a base operatively coupled to the proximal end of the robotarm, a plurality of motors disposed within the base, and a controlleroperatively coupled to the plurality of motors and programmed to permitthe robot arm to be freely moveable relative to the base responsive tomovement at the handle of the surgical instrument for performinglaparoscopic surgery using the surgical instrument. The robot arm mayinclude a proximal end, a distal end configured to be removably coupledto the surgical instrument, a plurality of links, and a plurality ofjoints between the proximal end and the distal end. In addition, theplurality of motors may be operatively coupled to at least some jointsof the plurality of joints.

The controller may be programmed to: measure a position of the distalend of the robot arm; determine a point of entry of the surgicalinstrument into the patient by determining a point of intersection of aplurality of virtual lines parallel to the longitudinal axis of thesurgical instrument as the position of the distal end of the robot armmoves relative to the point of entry; calculate a compensation forcerequired to compensate for gravity of the surgical instrument based onthe position of the distal end of the robot arm, the point of entry, andone or more instrument parameters stored in a memory of the controller;and apply torque to the at least some joints of the plurality of jointsof the robot arm via the plurality of motors based on the compensationforce to compensate for gravity of the surgical instrument duringoperation of the co-manipulation surgical system. Moreover, thecontroller may be programmed to cause the robot arm to maintain a staticposition in a passive mode responsive to determining that movement ofthe robot arm due to movement at the handle of the surgical instrumentis less than a predetermined amount for at least a predetermined dwelltime period.

The controller further may be programmed to: measure a change ofposition of the distal end of the robot arm responsive to an externalforce; calculate a hold force required to maintain the static positionof the robot arm based on the position of the distal end of the robotarm, the point of entry, and the one or more instrument parameters; andapply torque to the at least some joints of the plurality of joints ofthe robot arm via the plurality of motors based on the compensationforce and the hold force to maintain the static position of the robotarm in the passive mode. The controller may be programmed to calculate aforce applied by the surgical instrument to the patient at the point ofentry based on the compensation force, the hold force, the one or moreinstrument parameters, and the point of entry. In addition, thecontroller may be programmed to calculate a force applied to theoperating end of the surgical instrument based on the compensationforce, the hold force, the one or more instrument parameters, and thepoint of entry.

The controller may be programmed to switch from the passive mode to aco-manipulation mode responsive to determining that the hold forceexceeds a predetermined threshold. For example, the controller may beprogrammed to permit the robot arm to be freely moveable in theco-manipulation mode responsive to movement at the handle of thesurgical instrument for performing laparoscopic surgery using thesurgical instrument while compensating for gravity of the surgicalinstrument. The system further may include a graphical user interfaceoperatively coupled to the controller. The graphical user interface maydisplay at least one of a force applied by the surgical instrument tothe patient at the point of entry, an external force applied to theoperating end of the surgical instrument, the predetermined threshold,or the one or more instrument parameters. In addition, the graphicaluser interface may permit a user to adjust the predetermined threshold.

The one or more instrument parameters may include at least one ofinstrument type, mass, center of mass, length, or instrument shaftdiameter. In addition, the controller may be programmed to calculate themass of the surgical instrument by executing a calibration routine. Thecontroller may be programmed to cause the base to move in at least onedegree of freedom. For example, the system further may include aplatform coupled to the base, such that the controller may causevertical and horizontal movement of the base relative to the platform.In addition, the system further may include one or more sensorsoperatively coupled to one or more joints of the plurality of joints ofthe robot arm. The one or more sensors may be configured to measureposition of the one or more joints and to generate data indicative ofthe position of the one or more joints, such that the controller may beprogrammed to measure the position of the distal end of the robot armbased on the data. For example, the one or more sensors may be one ormore encoders disposed on the one or more joints of the plurality ofjoints of the robot arm. The plurality of encoders may be configured tomeasure angulation of corresponding links of the plurality of links atthe at least some joints, such that the controller may be programmed tomeasure the position of the distal end of the robot arm in 3D spacebased on the angulation measurements by the plurality of encoders.

In accordance with yet another aspect of the present disclosure, amethod for assisting with laparoscopic surgery using a robot armcomprising a proximal end, a distal end configured to be removablycoupled to a surgical instrument, a plurality of links, and a pluralityof joints between the proximal end and the distal end is provided. Themethod may include: measuring, via a controller operatively coupled to aplurality of motors operatively coupled to at least some joints of theplurality of joints, a position of the distal end of the robot arm;determining a point of entry of the surgical instrument into the patientby determining a point of intersection of a plurality of virtual linesparallel to the longitudinal axis of the surgical instrument as theposition of the distal end of the robot arm moves relative to the pointof entry; calculating a compensation force required to compensate forgravity of the surgical instrument based on the position of the distalend of the robot arm, the point of entry, and one or more instrumentparameters stored in a memory of the controller; and applying torque tothe at least some joints of the plurality of joints of the robot arm viathe plurality of motors based on the compensation force to compensatefor gravity of the surgical instrument during the laparoscopic surgery.The controller may be programmed to permit the robot arm to be freelymoveable relative to a base operatively coupled to the proximal end ofthe robot arm responsive to movement at the handle of the surgicalinstrument for performing laparoscopic surgery using the surgicalinstrument while compensating for gravity of the surgical instrument.

The method further may include causing the robot arm to maintain astatic position in a passive mode responsive to determining thatmovement of the robot arm due to movement at the handle of the surgicalinstrument is less than a predetermined amount for at least apredetermined dwell time period. Moreover, the method may include:measuring a change of position of the distal end of the robot armresponsive to an external force; calculating a hold force required tomaintain the static position of the robot arm based on the position ofthe distal end of the robot arm, the point of entry, and the one or moreinstrument parameters; and applying torque to the at least some jointsof the plurality of joints of the robot arm via the plurality of motorsbased on the compensation force and the hold force to maintain thestatic position of the robot arm in the passive mode.

In addition, the method may include calculating a force applied by thesurgical instrument to the patient at the point of entry based on thecompensation force, the hold force, the one or more instrumentparameters, and the point of entry. Moreover, the method may includecalculating a force applied to an operating end of the surgicalinstrument based on the compensation force, the hold force, the one ormore instrument parameters, and the point of entry. The method furthermay include switching from the passive mode to a co-manipulation moderesponsive to determining that the hold force exceeds a predeterminedthreshold, wherein the robot arm is freely moveable in theco-manipulation mode responsive to movement at the handle of thesurgical instrument for performing laparoscopic surgery using thesurgical instrument while compensating for gravity of the surgicalinstrument.

In accordance with one aspect of the present disclosure, a device forremovably coupling a surgical instrument having an elongated shaft to adistal end of a robot arm of a co-manipulation surgical system to assistwith laparoscopic surgery performed using the surgical instrument,wherein the distal end of the robot arm comprises a coupler interfacehaving a protrusion is provided. The device may include a coupler bodyconfigured to be removably coupled to the coupler interface and to theelongated shaft of the surgical instrument. For example, the couplerbody may include a groove configured to receive the protrusion of thecoupler interface, an opening sized and shaped to receive the elongatedshaft therein, and a switch configured to transition between an unlockedposition and a locked position. The switch may include an engagementportion configured to engage with the elongated shaft when the elongatedshaft is disposed within the opening and the switch is in the lockedposition to thereby secure the elongated shaft within the opening.Accordingly, when the coupler body is coupled to the coupler interface,the elongated shaft is disposed within the opening, and the switch is inthe locked position, the robot arm may be configured to be freelymoveable responsive to movement at the handle of the surgicalinstrument.

Moreover, when the elongated shaft is disposed within the opening andthe switch is in the locked position, the engagement portion may beconfigured to apply a friction force against the elongated shaft, suchthat the friction force permits rotational movement of the elongatedshaft within the opening, while prohibiting translational movement ofthe elongated shaft relative to the coupler body. The coupler body mayinclude one or more tapered surfaces configured to guide the elongatedshaft into the opening. In addition, the one or more tapered surfacesmay be configured to facilitate self-alignment of the distal end of therobot arm relative to the surgical instrument by causing the couplerbody and the coupler interface to rotate to align the opening with theelongated shaft as the elongated shaft is inserted along the one or moretapered surfaces into the opening.

The device further may include a clamp configured to transition betweenan unlocked state where the opening is configured to receive theelongated shaft and a locked state where the clamp secures the elongatedshaft within the opening. The clamp may be configured to be biasedtoward the locked state. Moreover, at least a portion of the clamp mayinclude a tapered surface configured to guide the elongated shaft intothe opening and to facilitate transitioning of the clamp from the lockedstate to the unlocked state responsive to a force applied to the taperedsurface by the elongated shaft as the elongated shaft is inserted intothe opening. The coupler body further may include a holder slidablydisposed within the coupler body. The holder may include a friction padconfigured to define at least a portion of the opening. In addition, theholder may be configured to be biased in a direction toward the openingsuch that, when the elongated shaft is disposed within the opening, thefriction pad is configured to engage with the elongated shaft.

In addition, the coupler interface may include a repulsion magnet, andthe holder may include a magnet, such that the repulsion magnet may beconfigured to apply a magnetic force to the magnet to thereby cause theholder to be biased in the direction toward the opening. For example,the holder may include a harness configured to be coupled to the magnet.The harness may be sized and shaped to be slidably disposed within achannel of the coupler body. The friction pad may be configured to applya friction force against the elongated shaft when the elongated shaft isdisposed within the opening and the switch is in the locked position,such that the friction force may be configured to permit rotationalmovement of the elongated shaft within the opening, while prohibitingtranslational movement of the elongated shaft relative to the couplerbody.

Moreover, the clamp may be pivotally coupled to the coupler body via arod, and the holder may include one or more cradles coupled to thefriction pad. Each of the one or more cradles may include a channelsized and shaped to slidably receive the rod therethrough, such that theholder may be configured to be slidably disposed within the coupler bodyalong the rod. In some embodiments, the holder may be coupled to acompression spring, such that the compression spring may be configuredto apply a spring force to the holder to thereby cause the holder to bebiased in the direction toward the opening. In addition, the clamp mayinclude a handle portion configured to be actuated to transition theclamp from the locked state to the unlocked state, and the switch mayinclude a handle configured to be actuated to transition the switchbetween the unlocked position and the locked position.

The protrusion may include one or more indentations, and the couplerbody may include one or more locking arms configured to transitionbetween a locked configuration where at least a portion of the one ormore locking arms extend within the groove of the coupler body, and anunlocked configuration where the one or more locking arms do not extendwithin the groove of the coupler body. Accordingly, the protrusion ofthe coupler interface may be configured to be received by the groove ofthe coupler body when the one or more locking arms are in the unlockedconfiguration, and the at least a portion of the one or more lockingarms may extend within the one or more indentations of the protrusionwhen the protrusion is disposed within the groove and the locking armsare in the locked configuration to thereby secure the coupler body tothe coupler interface. In addition, the protrusion of the couplerinterface may have a first geometry and the groove of the coupler bodymay have a second geometry corresponding to the first geometry, suchthat, when the protrusion is received by the groove, rotational movementbetween the coupler body and the coupler interface is prohibited.

The one or more locking arms may be biased toward the lockedconfiguration. Moreover, each of the one or more locking arms mayinclude a handle portion configured to be actuated to transition the oneor more locking arms from the locked configuration to the unlockedconfiguration. In addition, the coupler body may configured such that asterile drape is configured to be disposed between the coupler body andthe coupler interface when the coupler body is coupled to the couplerinterface.

In some embodiments, the coupler interface may include a ferrous rodextending along a longitudinal axis of the coupler interface, and one ormore sensors configured to measure a magnetic field of the ferrous rod.Accordingly, when the coupler body is coupled to the coupler interface,the magnet may induce a magnetic field in the ferrous rod, wherein astrength of the induced magnetic field is proportional to the positionof the magnet relative to the ferrous rod. Moreover, when the couplerbody is coupled to the coupler interface and the elongated shaft is notdisposed within the opening, the magnetic force of the repulsion magnetmay cause the magnet to move to a predefined position within the couplerbody, such that the magnetic field induced in the ferrous rod by themagnet at the predefined position may be indicative of the coupler bodybeing coupled to the coupler interface without the surgical instrumentcoupled to the coupler body. Additionally, when the coupler body iscoupled to the coupler interface and the elongated shaft is disposedwithin the opening, a force applied to the holder by the elongated shaftvia the friction pad and the magnetic force of the repulsion magnet maycause the magnet to move to a predefined position within the couplerbody, such that the magnetic field induced in the ferrous rod by themagnet at the predefined position may be indicative of the coupler bodybeing coupled to the coupler interface and the surgical instrument beingcoupled to the coupler body. In addition, the magnetic field induced inthe ferrous rod by the magnet at the predefined position may beindicative of a size of the surgical instrument coupled to the couplerbody.

In accordance with another aspect of the present disclosure, anotherco-manipulation surgical system is provided. The co-manipulationsurgical system may include a robot arm comprising a proximal end, adistal end configured to be removably coupled to the surgicalinstrument, a plurality of links, and a plurality of joints, a couplerinterface disposed at the distal end of the robot arm, and a couplerbody configured to be removably coupled to the coupler interface and tothe elongated shaft of the surgical instrument. The coupler interfacemay include a ferrous rod and one or more sensors, e.g., one or moreHall effect sensors, configured to measure a magnetic field of theferrous rod. The coupler body may include a magnet slidably disposedwithin the coupler body. For example, the magnet may be configured toinduce a magnetic field in the ferrous rod based on a position of themagnet relative to the ferrous rod within the coupler body. In addition,the co-manipulation surgical system may include a controller operativelycoupled to the robot arm and the one or more sensors, such that thecontroller may be programmed to determine whether the coupler body iscoupled to the coupler interface based on the magnetic field of theferrous rod measured by the one or more sensors.

Moreover, the controller may be programmed to determine whether thesurgical instrument is coupled to the coupler body when the coupler bodyis coupled to the coupler interface based on the magnetic field of theferrous rod measured by the one or more sensors. In addition, thecontroller may be programmed to identify a size of the surgicalinstrument coupled to the coupler body based on the magnetic field ofthe ferrous rod measured by the one or more sensors. The magnet may bebiased in a direction away from coupler interface when the coupler bodyis coupled to the coupler interface.

When the coupler body is coupled to the coupler interface and nosurgical instrument is coupled to the coupler body, the magnet may bedisposed within a predefined position within the coupler body, such thatthe controller may be configured to determine that the coupler body iscoupled to the coupler interface without the surgical instrument coupledto the coupler body based on the magnetic field induced in the ferrousrod by the magnet at the predefined position measured by the one or moresensors. Moreover, when the coupler body is coupled to the couplerinterface and the surgical instrument is coupled to the coupler body,the magnet may be disposed within a predefined position within thecoupler body, such that the controller may be configured to determinethat the coupler body is coupled to the coupler interface and thesurgical instrument is coupled to the coupler body based on the magneticfield induced in the ferrous rod by the magnet at the predefinedposition measured by the one or more sensors.

The coupler body may include a holder slidably disposed within thecoupler body. The holder may be configured to be coupled to the magnetand may include a friction pad configured to engage with the elongatedshaft when the elongated shaft is coupled to the coupler body and thecoupler body is coupled to the coupler interface, such that, when theelongated rod is coupled to the coupler body, the elongated rod appliesa force to the friction pad to thereby move the magnet to the predefinedposition within the coupler body via the holder. The coupler interfacemay include a repulsion magnet configured to apply a magnetic force tothe magnet to bias the magnet in the direction away from couplerinterface when the coupler body is coupled to the coupler interface.Alternatively, the magnet may be coupled to a compression springconfigured to bias the magnet in the direction away from couplerinterface. The controller may be configured to permit the robot arm tobe freely moveable responsive to movement at the handle of the surgicalinstrument for performing laparoscopic surgery using the surgicalinstrument when the surgical instrument is coupled to the couplerinterface via the coupler body.

In accordance with another aspect of the present disclosure, a methodfor assisting with laparoscopic surgery using a robot arm comprising aproximal end, a distal end having a coupler interface configured to beremovably coupled to a surgical instrument via a coupler body, aplurality of links, and a plurality of joints is provided. The methodmay include: measuring, via one or more sensors, a magnetic field of aferrous rod extending within the distal end of the robot arm; anddetermining, via a controller operatively coupled to the one or moresensors, whether the coupler body is coupled to the coupler interface atthe distal end of the robot arm based on the magnetic field of theferrous rod measured by the one or more sensors, wherein the couplerbody comprises a magnet slidably disposed therein, such that themagnetic field of the ferrous rod measured by the one or more sensorsvaries based on a position of the magnet relative to the ferrous rod,and wherein the magnet is biased in a direction away from the couplerinterface when the coupler body is coupled to the coupler interface.

In accordance with another aspect of the present disclosure, anotherco-manipulation surgical system is provided. The co-manipulationsurgical system may include a robot arm comprising a proximal endoperatively coupled to a base, a distal end configured to be removablycoupled to the surgical instrument, a plurality of links, and aplurality of joints, a platform coupled to the base, the platformconfigured to move the base in at least one degree of freedom, aplurality of motors operatively coupled to at least some joints of theplurality of joints, and one or more sensors configured to collectsensor data comprising at least one of 3D depth data or pixel imagedata. The co-manipulation surgical system further may include acontroller operatively coupled to the robot arm and the one or moresensors, and configured to permit the robot arm to be freely moveableresponsive to movement at the handle of the surgical instrument forperforming laparoscopic surgery using the surgical instrument, thecontroller programmed to: identify at least one of a position ororientation of one or more objects, e.g., a surgical bed, within anoperating room based on the sensor data from the one or more sensors;estimate a relative distance between the one or more objects and atleast one of the base or the robot arm as the at least one of the baseor the robot arm moves within the operating room; and apply torque orimpedance to the at least some joints of the plurality of joints of therobot arm via the plurality of motors to reposition or stop movement ofthe robot arm to avoid a collision between the one or more objects andthe at least one of the base or the robot arm if the estimated relativedistance approaches a predetermined threshold.

Moreover, the controller may be configured to: detect a movement at thedistal end of the robot arm in a first direction responsive to a firstforce applied to the distal end of the robot arm by a user; cause theplatform to move the base in the first direction responsive to thedetection of the movement at the distal end of the robot arm in thefirst direction; and cause the platform to stop movement of the base inthe first direction if the first force applied to the distal end of therobot arm by the user falls below a predetermined threshold. Thecontroller may be configured to cause the platform to move the base inthe first direction if the first force applied to the distal end of therobot arm exceeds a predetermined force threshold. The controller may beconfigured to identify a plane of the one or more objects within theoperating room based on the sensor data from the one or more sensors,and estimate the relative distance between the one or more objects andthe at least one of the base or the robot arm based on the plane of theone or more objects. In addition, the controller may be configured to:determine a type of laparoscopic surgery to be performed; identify atleast one of a position or orientation of a trocar port based on thesensor data from the one or more sensors; and apply torque to the atleast some joints of the plurality of joints of the robot arm via theplurality of motors to automatically position the robot arm in apredetermined configuration relative to the trocar port based on thetype of laparoscopic surgery to be performed.

In accordance with another aspect of the present disclosure, anotherco-manipulation surgical system is provided. The co-manipulationsurgical system may include a robot arm comprising a proximal end, adistal end configured to be removably coupled to the surgicalinstrument, a plurality of links, and a plurality of joints, one or moresensors configured to collect sensor data comprising at least one of 3Ddepth data or pixel image data, and a controller operatively coupled tothe robot arm and the one or more sensors, and configured to permit therobot arm to be freely moveable responsive to movement at the handle ofthe surgical instrument for performing laparoscopic surgery using thesurgical instrument. The controller may be programmed to: determine atleast one of a position or orientation of a trocar port relative to therobot arm based on the sensor data from the one or more sensors; detectmovement of the trocar port based on the sensor data from the one ormore sensors when the operating end of the surgical instrument isinserted through the trocar port; and reposition the robot arm tomaintain a position of the operating end of the surgical instrumentrelative to the trocar port during the movement of the trocar port. Forexample, the controller may be configured to detect movement of thetrocar port responsive to movement of a surgical bed. Moreover, thecontroller may be configured to detect movement of the trocar portresponsive to movement of a patient’s body responsive to breathing bythe patient. In some embodiments, the controller may be configured tocause the distal end of the robot arm to retract the operating end ofthe surgical instrument within the trocar port prior to repositioningthe robot arm to maintain the position of the operating end of thesurgical instrument relative to the trocar port during the movement ofthe trocar port.

In accordance with another aspect of the present disclosure, aco-manipulation surgical system that may be configured to calibrate anew robot arm is provided. The co-manipulation surgical system mayinclude a robot arm comprising a proximal end configured to beremoveably coupled to a cart, a distal end configured to be removablycoupled to the surgical instrument, a plurality of links, and aplurality of joints, an optical scanner configured to measure depthdata, and a controller operatively coupled to the robot arm and theoptical scanner, and configured to permit the robot arm to be freelymoveable responsive to movement at the handle of the surgical instrumentfor performing laparoscopic surgery using the surgical instrument. Thecontroller may be programmed to: cause the robot arm to move in anintended predefined pattern of movement relative to the cart inaccordance with a preprogrammed routine; compare depth data from theoptical scanner indicative of an actual movement of the robot armresponsive to the preprogrammed routine with the intended predefinedpattern of movement, and generate a degree of error indicative of adeviation between the actual movement of the robot arm and the intendedpredefined pattern of movement; and execute an optimization algorithmconfigured to reduce the degree of error such that the deviation betweenthe actual movement of the robot arm and the intended predefined patternof movement decreases. The controller may be configured to permit therobot arm to be freely moveable responsive to movement at the handle ofthe surgical instrument for performing laparoscopic surgery using thesurgical instrument when the degree of error falls below a predeterminedthreshold.

In accordance with another aspect of the present disclosure, aco-manipulation surgical system that may be configured to track surgicalinstruments and overlay a virtual menu on a video feed is provided. Theco-manipulation surgical system may include a robot arm comprising aproximal end, a distal end configured to be removably coupled to thesurgical instrument, a plurality of links, and a plurality of joints,and a controller operatively coupled to the robot arm and a laparoscopeconfigured to generate a video feed. The controller may be configured topermit the robot arm to be freely moveable responsive to movement at thehandle of the surgical instrument for performing laparoscopic surgeryusing the surgical instrument. Moreover, the controller may beprogrammed to: cause a virtual menu to overlay on the video feeddisplayed on a display; track movement of the operating end of thesurgical instrument responsive to movement at the handle of the surgicalinstrument within the video feed to detect one or more predeterminedgestural patterns of movement of the operating end; and actuate afunction of the co-manipulation surgical system associated with thevirtual menu based on the detection of the one or more predeterminedgestural patterns of movement of the operating end relative to thevirtual menu. The virtual menu may include one or more menu optionsoverlaid in at least one corner of the video feed.

The function of the co-manipulation surgical system associated with thevirtual menu may include, for example, adjusting a holding forcethreshold required to be exceeded to cause the robot arm to switch froma passive mode where the controller causes the robot arm to maintain astatic position to a co-manipulation mode where the controller permitsthe robot arm to be freely moveable responsive to movement at the handleof the surgical instrument for performing laparoscopic surgery using thesurgical instrument. In addition, the function of the co-manipulationsurgical system associated with the virtual menu may include actuationof an assisted scope mode where the controller causes the laparoscope toautomatically adjust at least one of field of view or position to assistwith the laparoscopic surgery. The controller may be configured to causethe virtual menu to overlay on the video feed displayed on the displayresponsive to user input received via a graphical user interfaceoperatively coupled to the controller, responsive to a voice command bya user, and/or responsive to actuation of an actuator disposed on therobot arm. In addition, the controller may be configured to trackmovement of the operating end of the surgical instrument responsive touser input received via a graphical user interface operatively coupledto the controller, responsive to a voice command by a user, and/orresponsive to actuation of an actuator disposed on the robot arm.

In accordance with another aspect of the present disclosure, aco-manipulation surgical system that may be configured to provideindications via haptic feedback is provided. The co-manipulationsurgical system may include a robot arm comprising a proximal end, adistal end configured to be removably coupled to the surgicalinstrument, a plurality of links, and a plurality of joints, and acontroller operatively coupled to the robot arm. The controller may beprogrammed to: cause the robot arm to automatically switch between aco-manipulation mode wherein the controller permits the robot arm to befreely moveable responsive to movement at the handle of the surgicalinstrument for performing laparoscopic surgery using the surgicalinstrument, and a passive mode where the controller causes the robot armto maintain a static position; and cause a vibration at the distal endof the robot arm, the vibration indicative of the robot arm switchingfrom the co-manipulation mode to the passive mode. For example, thevibration may be configured to be perceivable by a user holding thehandle of the surgical instrument, while causing negligible movement atthe operating end of the surgical instrument. In addition, thecontroller may be configured to cause a second vibration at the distalend of the robot arm when the surgical instrument is coupled to distalend of the robot arm, wherein the second vibration may be indicativethat the surgical instrument is coupled to the distal end of the robotarm.

The controller may be configured to cause the robot arm to switch to thepassive mode responsive to determining that movement of the robot armdue to movement at the handle of the surgical instrument is less than apredetermined amount for at least a predetermined dwell time period.Moreover, the controller may be configured to cause the robot arm toswitch to the co-manipulation mode responsive to determining that forceapplied at the robot arm due to force applied at the handle of thesurgical instrument exceeds a predetermined threshold. The controllermay be configured to apply a first impedance to the robot arm in theco-manipulation mode to account for weight of the surgical instrumentand the robot arm. In addition, the controller may be configured togenerate an audible alert indicative of the robot arm switching from theco-manipulation mode to the passive mode. The robot arm further mayinclude a base operatively coupled to the proximal end of the robot arm,and the system further may include a plurality of motors disposed withinthe base, wherein the plurality of motors are operatively coupled to atleast some joints of the plurality of joints. Accordingly, thecontroller may be programmed to actuate at least one motor of theplurality of motors to cause the vibration at the distal end of therobot arm.

In accordance with another aspect of the present disclosure, aco-manipulation surgical system that may be configured for automatedscope detection is provided. The co-manipulation surgical system mayinclude a robot arm comprising a proximal end, a distal end configuredto be removably coupled to the laparoscope, a plurality of links, and aplurality of joints, an optical scanner configured to measure depthdata, and a controller operatively coupled to the robot arm and theoptical scanner. The controller may be programmed to: compare movementof the laparoscope based on depth data from the optical scanner withmovement of the field of view of the laparoscope during the movement ofthe laparoscope based on the video feed collected from the operating endof the laparoscope; and identify a type of the laparoscope based on themovement of the field of view of the laparoscope during the movement ofthe laparoscope. The controller may be configured to execute apreprogrammed routine in a calibration mode to cause the movement of thelaparoscope in a predefined pattern of movement in accordance with thepreprogrammed routine. For example, the predefined pattern of movementmay include a circular motion. Alternatively, the movement of thelaparoscope may be responsive to movement at the handle of thelaparoscope by a user.

The type of laparoscope may be an angular degree of the operating end ofthe laparoscope. For example, the controller may be configured toidentify the type of the laparoscope as a flat tipped laparoscope whenthe movement of the laparoscope comprises a circular motion and themovement of the field of view of the laparoscope during the circularmovement of the laparoscope comprises a corresponding circular motion.Moreover, the controller may be configured to identify the type of thelaparoscope as a flat tipped laparoscope when the movement of thelaparoscope comprises a circular motion and the movement of the field ofview of the laparoscope during the circular movement of the laparoscopecomprises no change in depth of the field of view. In addition, thecontroller may be configured to identify the type of the laparoscope asan angled-tipped laparoscope when the movement of the laparoscopecomprises a circular motion and the movement of the field of view of thelaparoscope during the circular movement of the laparoscope comprises achange in depth of the field of view. The controller may be configuredto permit the robot arm to be freely moveable responsive to movement atthe handle of the laparoscope for performing laparoscopic surgery usingthe laparoscope.

In accordance with another aspect of the present disclosure, aco-manipulation surgical system to assist with laparoscopic surgeryperformed using a surgical instrument having a handle, an operating end,and an elongated shaft therebetween is provided. The system may includea robot arm comprising a proximal end, a distal end configured to beremovably coupled to the surgical instrument, a plurality of links, anda plurality of joints between the proximal end and the distal end, thedistal end of the robot arm comprising a coupler interface, and acoupler body configured to be removably coupled to the couplerinterface. The coupler body may include a lumen sized and shaped toreceive the elongated shaft of the surgical instrument therein, and maybe configured to transition between an open state where the elongatedshaft is slidably moveable within the lumen, and a closed state wherelongitudinal movement of the elongated shaft relative to the couplerbody is inhibited while rotational movement of the elongated shaftrelative to the coupler body is permitted responsive to movement at thehandle of the surgical instrument. Moreover, when the coupler body iscoupled to the coupler interface in the closed state, the robot arm maybe permitted to be freely moveable responsive to movement at the handleof the surgical instrument for performing laparoscopic surgery. In someembodiments, the coupler body may be disposable after a singlelaparoscopic surgery. Alternatively, the coupler body may besterilizeable, such that it may be reused for multiple surgicalprocedures.

The system further may include a switch configured to transition betweenan unlocked position and a locked position. The switch may include anengagement portion configured to engage with the elongated shaft whenthe elongated shaft is disposed within the lumen and the switch is inthe locked position to thereby secure the elongated shaft within thelumen. Accordingly, when the coupler body is coupled to the couplerinterface, the elongated shaft is disposed within the lumen, and theswitch is in the locked position, the robot arm may be configured to befreely moveable responsive to movement at the handle of the surgicalinstrument. Moreover, when the elongated shaft is disposed within thelumen and the switch is in the locked position, the engagement portionmay be configured to apply a friction force against the elongated shaft,the friction force configured to permit rotational movement of theelongated shaft relative to the coupler body, while inhibitinglongitudinal movement of the elongated shaft relative to the couplerbody. The switch may include a handle portion configured to be actuatedto transition the switch between the unlocked position and the lockedposition.

The coupler body further may include a holder slidably disposed withinthe coupler body. The holder may include a contact surface configured todefine at least a portion of the lumen, and may be configured to bebiased in a direction toward the lumen such that, when the elongatedshaft is disposed within the lumen, the contact surface is configured toengage with the elongated shaft. Additionally, the coupler interface mayinclude a repulsion magnet, and the holder may include a magnet, suchthat the repulsion magnet may be configured to apply a magnetic force tothe magnet to thereby cause the holder to be biased in the directiontoward the lumen. Moreover, the holder may include a harness configuredto be coupled to the magnet, the harness sized and shaped to be slidablydisposed within a channel of the coupler body. In addition, the contactsurface may be configured to apply a friction force against theelongated shaft when the elongated shaft is disposed within the lumenand the switch is in the locked position, such that the friction forcemay be configured to permit rotational movement of the elongated shaftrelative to the coupler body, while inhibiting translational movement ofthe elongated shaft relative to the coupler body.

The coupler body further may include a clamp configured to transitionbetween an unlocked state where the lumen is permitted to receive theelongated shaft and a locked state where the clamp secures the elongatedshaft within the lumen. The clamp may be pivotally coupled to thecoupler body via a rod, and may be configured to be biased toward thelocked state. Accordingly, the clamp further may include a handleportion configured to be actuated to cause the clamp to transition fromthe locked state to the unlocked state, e.g., to release the surgicalinstrument from the coupler body. Moreover, at least a portion of theclamp may include a tapered surface configured to guide the elongatedshaft into the lumen and to facilitate transitioning of the clamp fromthe locked state to the unlocked state responsive to a force applied tothe tapered surface by the elongated shaft as the elongated shaft isinserted into the lumen. In addition, the holder may include one or morecradles coupled to the contact surface, each of the one or more cradlescomprising a channel sized and shaped to slidably receive the rodtherethrough, such that the holder may be configured to be slidablydisposed within the coupler body along the rod.

The coupler interface further may include a protrusion, and the couplerbody may include a groove configured to receive the protrusion of thecoupler interface. For example, the protrusion may include one or moreindentations, and the coupler body may include one or more locking armsconfigured to transition between a locked configuration where at least aportion of the one or more locking arms extend within the groove of thecoupler body, and an unlocked configuration where the one or morelocking arms do not extend within the groove of the coupler body. Theprotrusion of the coupler interface may be configured to be received bythe groove of the coupler body when the one or more locking arms are inthe unlocked configuration, and at least a portion of the one or morelocking arms may extend within the one or more indentations of theprotrusion when the protrusion is disposed within the groove and thelocking arms are in the locked configuration to thereby secure thecoupler body to the coupler interface. The one or more locking arms maybe biased toward the locked configuration. In addition, each of the oneor more locking arms may include a handle portion configured to beactuated to transition the one or more locking arms from the lockedconfiguration to the unlocked configuration.

Moreover, the protrusion of the coupler interface may have a firstgeometry and the groove of the coupler body may have a second geometrycorresponding to the first geometry, such that, when the protrusion isreceived by the groove, rotational movement between the coupler body andthe coupler interface is prohibited. In some embodiments, the couplerinterface may include one or more additional protrusions having a firstgeometry, and the coupler body may include one or more additionalgrooves having a second geometry, such that, when the one or moreadditional protrusions are received by the one or more additionalgrooves, rotational movement between the coupler body and the couplerinterface is prohibited. In addition, the coupler body and the couplerinterface may be configured to receive a sterile drape therebetween,such that the sterile drape prevents contact between the surgicalinstrument and the robot arm during the laparoscopic surgery. Thecoupler body further may include one or more tapered surfaces configuredto guide the elongated shaft into the lumen and facilitateself-alignment of the distal end of the robot arm relative to thesurgical instrument by causing the coupler body and the couplerinterface to rotate to align the lumen with the elongated shaft as theelongated shaft is inserted along the one or more tapered surfaces intothe lumen.

In accordance with another aspect of the present disclosure, a methodfor assisting with laparoscopic surgery using the robot arm configuredto be removably coupled to a surgical instrument having a handle, anoperating end, and an elongated shaft therebetween is provided. Themethod may include: removably coupling a coupler body to a couplerinterface at a distal end of the robot arm; inserting the elongatedshaft of the surgical instrument into a lumen of the coupler body;transitioning the coupler body from an open state where the elongatedshaft is slidably moveable within the lumen to a closed state wherelongitudinal movement of the elongated shaft relative to the couplerbody is inhibited while rotational movement of the elongated shaftrelative to the coupler body is permitted responsive to movement at thehandle of the surgical instrument; and freely moving the robot arm bymoving the handle of the surgical instrument when the coupler body iscoupled to the coupler interface in the closed state to perform thelaparoscopic surgery.

For example, removably coupling the coupler body to the couplerinterface may include: actuating one or more locking arms of the couplerbody to transition the one or more locking arms from a lockedconfiguration where at least a portion of the one or more locking armsextend within a groove of the coupler body, to an unlocked configurationwhere the one or more locking arms do not extend within the groove;inserting a protrusion of the coupler interface within a groove of thecoupler body; and releasing the one or more locking arms to transitionthe one or more locking arms from the unlocked configuration to thelocked configuration, such that the at least a portion of the one ormore locking arms extend within one or more indentations of theprotrusion to thereby secure the coupler body to the coupler interface.Moreover, inserting the elongated shaft of the surgical instrumentwithin the lumen of the coupler body may include guiding the elongatedshaft along one or more tapered surfaces of the coupler body into thelumen, e.g., by rotating the coupler body and the coupler interface tofacilitate self-alignment of the lumen with the elongated shaft as theelongated shaft is inserted along the one or more tapered surfaces intothe lumen.

In addition, inserting the elongated shaft of the surgical instrumentwithin the lumen of the coupler body may include: actuating a clamp ofthe coupler body to transition the clamp from a locked state to anunlocked state where the lumen is permitted to receive the elongatedshaft; inserting the elongated shaft of the surgical instrument withinthe lumen; and releasing the clamp to transition the clamp from theunlocked state to the locked state, such that the clamp secures theelongated shaft within the lumen. Transitioning the coupler body fromthe open state to the closed state may include transitioning a switch ofthe coupler body from an unlocked position where the elongated shaft isslidably moveable within the lumen, to a locked position where anengagement portion of the switch engages with the elongated shaftdisposed within the lumen to thereby inhibit longitudinal movement ofthe elongated shaft relative to the coupler body while permittingrotational movement of the elongated shaft relative to the coupler body.The method further may include positioning a sterile drape between thecoupler body and the coupler interface prior to removably coupling thecoupler body to the coupler interface.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B illustrate a traditional laparoscopic procedureperformed by a surgeon and one or more assistants.

FIG. 2 illustrates an exemplary co-manipulation surgical systemconstructed in accordance with the principles of the present disclosure.

FIGS. 3A-3D illustrate an exemplary robot arm of the system of FIG. 2constructed in accordance with the principles of the present disclosure.

FIGS. 4A and 4B illustrate an exemplary wrist portion of the robot armof FIGS. 3A-3D constructed in accordance with the principles of thepresent disclosure.

FIG. 4C is a close-up view of an exemplary surgical instrument couplingmechanism of the wrist portion of FIGS. 4A and 4B.

FIG. 4D is a close-up view of an exemplary robot arm coupler interfaceof the surgical instrument coupling mechanism of FIG. 4C constructed inaccordance with the principles of the present disclosure.

FIGS. 5A and 5B illustrate an exemplary surgical instrument coupler bodyof the surgical instrument coupling mechanism of FIG. 4C constructed inaccordance with the principles of the present disclosure.

FIG. 6A illustrates an alternative exemplary surgical instrument couplerbody constructed in accordance with the principles of the presentdisclosure.

FIGS. 6B-6D illustrate attachment of the coupler body of FIG. 6A to asurgical retractor device in accordance with the principles of thepresent disclosure.

FIG. 7A illustrates another alternative exemplary surgical instrumentcoupler body constructed in accordance with the principles of thepresent disclosure.

FIGS. 7B-7D illustrate attachment of the coupler body of FIG. 7A to asurgical laparoscope device in accordance with the principles of thepresent disclosure.

FIGS. 8A and 8B illustrate the robot arms in a sterile-drape readyconfiguration.

FIGS. 9A and 9B illustrate the robot arms covered in a sterile drape.

FIGS. 10A-10D illustrate rotation of the shoulder link of the robot armin accordance with the principles of the present disclosure.

FIG. 11A illustrates an exemplary co-manipulation surgical system havingan optical scanner in accordance with the principles of the presentdisclosure, and FIG. 11B illustrates the optical scanner of FIG. 11A.

FIG. 11C illustrates an exemplary co-manipulation surgical system havinga plurality of optical scanners in accordance with the principles of thepresent disclosure.

FIG. 12 illustrates a user operating the co-manipulation surgical systemof FIG. 11A in accordance with the principles of the present disclosure.

FIG. 13A illustrates a field of view of the optical scanner during alaparoscopic surgical procedure, and FIG. 13B illustrates a depth map ofthe field of view the optical scanner of FIG. 13A.

FIG. 14 shows some example components that may be included in aco-manipulation robot platform in accordance with the principles of thepresent disclosure.

FIG. 15 is a flow chart illustrating operation of the co-manipulationsurgical system in accordance with the principles of the presentdisclosure.

FIG. 16 is a flow chart illustrating surgical instrument calibration ofthe co-manipulation surgical system in accordance with the principles ofthe present disclosure.

FIG. 17 is a flow chart illustrating operation of the robot arm inaccordance with the principles of the present disclosure.

FIGS. 18A and 18B are free-body diagrams illustrating forces applied tothe surgical instrument coupled to the robot arm during a laparoscopicsurgical procedure.

FIG. 19 is a table of example values related to some arrangements of apassive mode of the robot arm in accordance with the principles of thepresent disclosure.

FIG. 20 illustrates an example overview of some features andcapabilities of the co-manipulation surgical system in accordance withthe principles of the present disclosure.

FIG. 21 is a schematic overview of some electrical components andconnectivity of the co-manipulation surgical system in accordance withthe principles of the present disclosure.

FIG. 22 is a flow chart illustrating an example process of acquisitionand processing of data from an optical scanner and an exampleapplication of the data in accordance with the principles of the presentdisclosure.

FIG. 23 is a schematic overview of data flow of the co-manipulationsurgical system in accordance with the principles of the presentdisclosure.

FIG. 24 is another schematic overview of data flow the co-manipulationsurgical system in accordance with the principles of the presentdisclosure.

FIG. 25 is a schematic overview of data flow and output control of theco-manipulation surgical system in accordance with the principles of thepresent disclosure.

FIG. 26 is a schematic overview of data flow in a network ofco-manipulation surgical systems in accordance with the principles ofthe present disclosure.

FIGS. 27A-27D illustrate vertical and horizontal movement of the robotarms in accordance with the principles of the present disclosure.

FIGS. 28A-28D illustrate an exemplary graphical user interface of theco-manipulation surgical system.

FIG. 29 is a schematic of an alternative co-manipulation surgical systemconstructed in accordance with the principles of the present disclosure.

FIGS. 30A-43 illustrate various alternative surgical instrument couplingmechanisms constructed in accordance with the principles of the presentdisclosure.

FIGS. 44A and 44B illustrates another exemplary surgical instrumentcoupling mechanism constructed in accordance with the principles of thepresent disclosure.

FIG. 44C is a cross-sectional view of the surgical instrument couplingmechanism of FIG. 44A.

FIGS. 45A and 45B illustrate a coupler interface of the surgicalinstrument coupling mechanism of FIGS. 44A-44C.

FIG. 45C is a cross-sectional view of the coupler interface of FIG. 45B.

FIGS. 46A-46C illustrate a coupler body of the surgical instrumentcoupling mechanism of FIGS. 44A-44C.

FIG. 46D illustrates a magnet holder of the coupler body of FIGS.46A-46C.

FIG. 46E is a cross-sectional view of the coupler body of FIG. 46A.

FIG. 47A is a cross-sectional view of the coupler body of FIG. 46A whenthe coupler body is detached from the coupler interface.

FIG. 47B is a cross-sectional view of the surgical instrument couplingmechanism of FIG. 44A when the coupler body is coupled to the couplerinterface.

FIGS. 48A and 48B illustrate various views of the surgical instrumentcoupling mechanism of FIG. 44A when coupling the coupler body to thecoupler interface in accordance with the principles of the presentdisclosure.

FIG. 49 is a cross-sectional view of the surgical instrument couplingmechanism of FIG. 44A when a surgical instrument is coupled to thecoupler body.

FIGS. 50A and 50B illustrate various views of the surgical instrumentcoupling mechanism of FIG. 44A when coupling the surgical instrument tothe coupler body in accordance with the principles of the presentdisclosure.

FIG. 51 is a cross-sectional view of an alternative exemplary surgicalinstrument coupling mechanism when a surgical instrument is coupled tothe coupler body, constructed in accordance with the principles of thepresent disclosure.

FIGS. 52A to 52E illustrate an alternative exemplary surgical instrumentcoupling mechanism constructed in accordance with the principles of thepresent disclosure.

FIGS. 53A and 53B illustrate an alternative exemplary coupler bodyconstructed in accordance with the principles of the present disclosure.

FIG. 54 illustrates another alternative exemplary coupler bodyconstructed in accordance with the principles of the present disclosure.

FIG. 55 illustrates an exemplary virtual overlay of a graphical userinterface of the co-manipulation surgical system.

FIG. 56 illustrates a motorized joint of an alternative exemplary robotarm constructed in accordance with the principles of the presentdisclosure.

DETAILED DESCRIPTION

Disclosed herein are co-manipulation surgical robot systems forassisting an operator, e.g., a surgeon, in performing a surgicalprocedure, e.g., a laparoscopic procedure, and methods of use thereof.Currently, laparoscopic procedures typically require a surgeon and oneor more assistants. For example, as shown in FIG. 1A, during alaparoscopic procedure assistant A1 may be required to hold retractordevice 12 to expose tissue for surgeon S, while another assistant A2 maybe required to hold laparoscope device 10 to provide a field of view ofthe surgical space within the patient to surgeon S via a display (notshown) during the procedure. As shown in FIG. 1A, assistant A2 may berequired to hold laparoscope device 10 in an impractical position, e.g.,from between the arms of surgeon S while the surgeon actively operatesadditional surgical instruments, e.g., surgical instruments 14 and 16.As further shown in FIG. 1A, surgeon S may need to let go of surgicalinstrument 16 in order to guide/reposition laparoscope device 10 held byassistant A2 in order to achieve the field of view desired by thesurgeon.

As shown in FIG. 1B, rail-mounted orthopedic retractors 18 may be usedto hold one or more surgical instruments in position during thelaparoscopic procedure, in attempt to free hands of the surgeon and/orassistant for other tasks, as well as for stability. As shown in FIG.1B, first rail-mounted orthopedic retractor 18 a may include retractorend 20 a for engaging with and holding laparoscope device 10 in positionupon actuation of lock 22 a. For example, lock 22 a may be disengagedsuch that retractor 18 a may be manually positioned at a desiredlocation relative to the patient, and re-engaged to lock retractor 18 a,and accordingly laparoscopic device 10 coupled thereto, in the desiredposition. As shown in FIG. 1B, second rail-mounted orthopedic retractor18 b having retractor end 20 b may be used during the procedure toengage with and hold another surgical instrument in position uponactuation of lock 22 b. Thus, retractors 18 a and 18 b require extensivemanual interaction with locks 22 a and 22 b, and with retractors 18 aand 18 b themselves, to reposition and lock the respective tools inposition.

The co-manipulation surgical robot systems described herein providesuperior control and stability such that the surgeon and/or assistantmay seamlessly position various off-the-shelf surgical instruments asneeded, thus avoiding the workflow limitations inherent to both humanand mechanical solutions. For example, the robot arms of theco-manipulation surgical robot system may provide surgical assistance byholding a first surgical instrument, e.g., a laparoscope, via a firstrobot arm, and a second surgical instrument, e.g., a retractor, via asecond robot arm, stable throughout the procedure to provide an optimumview of the surgical site and reduce the variability of force applied bythe surgical instruments to the body wall at the trocar point. As willbe understood by a person having ordinary skill in the art, the robotsarms of the co-manipulation surgical robot systems described herein mayhold any surgical instrument, preferably having a long and thininstrument shaft, used for surgical procedures such as laparoscopicprocedures including, e.g., endoscopes/laparoscopes, retractors,graspers, surgical scissors, needle holders, needle drivers, clamps,suturing instruments, cautery tools, staplers, clip appliers, etc.

The co-manipulation surgical robot system further allows the surgeon toeasily maneuver both tools when necessary, providing superior controland stability over the procedure and overall safety. Any implementationsof the systems described herein enable a surgeon to directlyco-manipulate instruments while remaining sterile at the patientbedside. For example, the system may include two robot arms that may beused by the surgeon to hold both a laparoscope and a retractor. During asurgical procedure, the system may seamlessly reposition eitherinstrument to provide optimal visualization and exposure of the surgicalfield. Both instruments may be directly coupled to the robot arms of thesystem and the system may constantly monitor and record the position ofthe two instruments and/or the two robot arms throughout the procedure.Moreover, the system may record information such as the position andorientation of surgical instruments attached to the robot arm, sensorreadings related to force(s) applied at proximal and distal ends of thesurgical instruments attached to robot arms, force required to hold eachinstrument in position, endoscopic video streams, algorithm parameters,operating room 3D stream captured with an optical scanning device,including, e.g., position(s) of surgical entry port(s), position andmovements of the surgeon’s hands, surgical instrument(s) position andorientation, whether or not attached to robot arms, patient position,and patient table orientation and height.

Such data may be used to develop a database of historical data that maybe used to develop the algorithms used in some implementations tocontrol one or more aspects of an operation of the system. In addition,such data may be used during a procedure to control of one or moreaspects of an operation of the system per one or more algorithms of thesystem. For example, the data may be used to assess a level of fatigueof a user of the system.

As the operator manipulates a robot arm of the co-manipulation surgicalrobot system by applying movement to the surgical instrument coupled tothe robot arm, the system may automatically transition the robot armbetween various operational modes upon determination of predefinedconditions. For example, the system may transition the robot arm to apassive mode responsive to determining that movement of the robot armdue to movement at the handle of the surgical instrument is less than apredetermined amount for at least a predetermined dwell time period,such that in the passive mode, the robot arm maintains a staticposition, e.g., to prevent damage to the equipment and/or injury to thepatient. Additionally, the system may transition the robot arm to aco-manipulation mode responsive to determining that force applied at therobot arm due to force applied at the handle of the surgical instrumentexceeds a predetermined threshold, such that in the co-manipulationmode, the robot arm is permitted to be freely moveable responsive tomovement at the handle of the surgical instrument for performinglaparoscopic surgery using the surgical instrument, while a firstimpedance is applied to the robot arm in the co-manipulation mode toaccount for weight of the surgical instrument and the robot arm.Moreover, the system may transition the robot arm to a haptic moderesponsive to determining that at least a portion of the robot arm isoutside a predefined haptic barrier, such that in the haptic mode, asecond impedance greater than the first impedance is applied to therobot arm, thereby making movement of the robot arm responsive tomovement at the handle of the surgical instrument more viscous in thehaptic mode than in the co-manipulation mode. The system further maytransition the robot arm to a robotic assist mode responsive todetecting various conditions that warrant automated movement of therobot arm to guide the surgical instrument attached thereto, e.g., alonga planned trajectory or to avoid a collision with another object orperson in the surgical space.

Referring now to FIG. 2 , co-manipulation surgical robot system 200 isprovided. As shown in FIG. 2 , system 200 may include platform 100,e.g., a surgical cart, sized and shaped to support or more robot arms300, e.g., robot arm 300 a and robot arm 300 b, each of robot arms 300having surgical instrument coupler interface 400 for removably couplingto a surgical instrument, and a computing system operatively coupled toplatform 100 and robot arms 300. As shown in FIG. 2 , system 200 furthermay include graphical user interface display 110 for displayingoperational information as well as receiving user input.

In addition, each of robot arms 300 further may include indicators 334for visually indicating the operational mode associated with therespective robot arm in real-time. For example, indicators 334 may bepositioned on at least the elbow joint of the robot arm. Additionally oralternatively, indicators 334 may be placed elsewhere on system 200,e.g., on platform 100, on display 110, etc. Moreover, indicators 334 mayinclude lights, e.g., LED lights, that may illuminate in a variety ofdistinct colors and in distinct patterns, e.g., solid on or blinking.For example, each operational mode of system 200 may be associated witha uniquely colored light, such as red, yellow, blue, green, purple,white, orange, etc. Accordingly, indicators 334 may indicate atransition from one operational mode to another operational mode.Additionally or alternatively, transitions from one operational mode toanother operational mode may be indicated to a user via haptic feedback,e.g., a vibration delivered to the distal end of robot arm 300, andaccordingly to the surgical instrument coupled thereto. For example, thedistal end of robot arm 300 may vibrate as robot arm 300 transitionsfrom co-manipulation mode to static mode to assure the user that robotarm 300 is in static mode and will remain in position upon release bythe user. Additionally or alternatively, an audible alert may be emittedto indicate to the user when robot arm 300 transitions from oneoperational mode to another operational mode.

As shown in FIG. 2 , platform 100 may include one or more stages coupledto the base portion of one or more robot arms, e.g., base portion 302 aof robot arm 300 a and base portion 302 b of robot arm 300 b, forproviding movement to the respective robot arm, e.g., in at least thehorizontal and vertical directions relative to platform 100. Each stagemay include vertical extenders, e.g., vertical extender 106 a orvertical extender 106 b, for independently moving robot arm 300 a orrobot arm 300 b, respectively, vertically relative to platform 100, andhorizontal extenders, e.g., horizontal extender 108 a or horizontalextender 108 b, for independently moving robot arm 300 a or robot arm300 b, respectively, horizontally relative to platform 100, to therebypermit the operator flexibility in positioning robot arms 300 relativeto the patient.

Moreover, platform 100 may include a plurality of wheels 104, e.g.,castor wheels, to provide mobility of platform 100, and accordingly,robot arms 300, within the operating room. Wheels 104 may each include abraking mechanism which may be actuated to prevent movement of platform100 via wheels 104. Accordingly, platform 100 may independently moveeach of robot arm 300 a and robot arm 300 b in any direction, includinga first or vertical direction toward and away from the floor, a secondor horizontal direction toward and away from the patient, and/or a thirddirection or horizontal direction along a length of the patient. In someembodiments, platform 100 may move robot arm 300 a and robot arm 300 bin the same direction simultaneously, and further may cause rotationmovement of robot arm 300 a and robot arm 300 b. When ready foroperation, platform 100 may be moved to a desired position at the sideof the patient bed and locked in place via wheels 104, and the verticaland horizontal positions of robot arms 300 a and 300 b may be adjustedto an optimum position relative to the patient for the procedure viavertical extenders 106 a, 106 b and horizontal extenders 108 a, 108 b,responsive to user input received by graphical user interface display110. As described in further detail below, platform 100 mayautomatically move robot arm 300 a and robot arm 300 b responsive todetection of, e.g., potential collisions with other objects and/orpersons within the operating room and/or user input applied via therobot arms, during a laparoscopic procedure and/or during setup of therobot arms.

Surgical robot system 200 is configured for co-manipulation, such thatsystem 200 may assist the user or operator, e.g., a surgeon and/orsurgical assistant, by permitting the user to freely move robot arm 300a and/or robot arm 300 b due to manipulation of one or more surgicalinstruments coupled with the robot arms in response to force inputsprovided by the user to the surgical instruments. Accordingly, system200 may be configured so that it is not controlled remotely, such thatrobot arms 300 move directly responsive to movement of the surgicalinstrument coupled thereto by the operator, while compensating for themass of the surgical instrument and of the respective robot arm andproviding localized impedance along the robot arm, thereby increasingthe accuracy of the movements or actions of the operator as the operatormanipulates the surgical instrument.

System 200 may be particularly useful in laparoscopic surgicalprocedures and/or other surgical procedures that utilize long and thininstruments that may be inserted, e.g., via cannulas, into the body of apatient to allow surgical intervention. As will be understood by aperson having ordinary skill in the art, system 200 may be used for anydesired or suitable surgical operation. Moreover, system 200 may be usedin conjunction or cooperation with video monitoring provided by one ormore cameras and/or one or more endoscopes so that an operator of system200 may view and monitor the use of the instrument coupled with robotarms 300 a, 300 b via respective coupler interfaces 400 a, 400 b. Forexample, robot arm 300 a may be removeably coupled with and manipulatean endoscope, while robot arm 300 b may be may be removeably coupledwith and manipulate a surgical instrument.

Referring now to FIGS. 3A to 3D, a surgical support arm, e.g., robot arm300, is provided. As described above, system 200 may include a pluralityof robot arms, e.g., robot arm 300 a and robot arm 300 b. however, aseach robot arm may be constructed identically, only a single robot armis described with regard to FIGS. 3A to 3D for brevity, collectively asrobot arm 300. Aspects of the robot arms described herein may utilizestructures from U.S. Pat. No. 10,118,289 to Louveau, the entire contentsof which are incorporated herein by reference. Robot arm 300 may includea plurality of arm segments/links and a plurality of articulation joints106 extending from a base portion. For example, robot arm 300 mayinclude a base portion, a shoulder portion, an elbow portion, and awrist portion, thereby mimicking the kinematics of a human arm. As shownin FIG. 3A, robot arm 300 may include a base, which includes baseportion 302 rotatably coupled to shoulder portion 304 at base joint 303.For example, shoulder portion 304 may sit on top of base portion 302,and may be rotated relative to base portion 302 about axis Q1 at basejoint 303. In some embodiments, robot arms 300 may be interchanged,swapped, or coupled with the base in any desired arrangement.

Robot arm 300 further may include shoulder link 305, which includesproximal shoulder link 306 rotatably coupled to distal shoulder link308. A proximal end of proximal shoulder link 306 may be rotatablycoupled to shoulder portion 304 of the base at shoulder joint 318, suchthat proximal shoulder link 306 may be rotated relative to shoulderportion 304 about axis Q2 at shoulder joint 318. As shown in FIG. 3A,axis Q2 may be perpendicular to axis Q1. The distal end of proximalshoulder link 306 may be rotatably coupled to the proximal end of distalshoulder link 308 at joint 320, such that distal shoulder link 308 maybe rotated relative to proximal shoulder link 306 about axis Q3 at joint320. As shown in FIG. 3A, axis Q3 may be parallel to the longitudinalaxis of shoulder link 305. In addition, robot arm 300 may includeactuator 330, e.g., a lever, button, or switch, operatively coupled todistal shoulder link 308 and/or proximal shoulder link 306, such thatdistal shoulder link 308 may only be rotated relative to proximal shouldlink 306 upon actuation of actuator 330. Accordingly, axis Q3 may be a“setup” axis, such distal shoulder link 308 may be rotated and fixedrelative to proximal shoulder link 306 during a setup stage prior tooperating stage where robot arm 300 is used in a surgical procedure, asdescribed in further detail with regard to FIGS. 10A to 10D.

In some embodiments, upon actuation of actuator 330, distal shoulderlink 308 may be manually rotated in predefined increments relative toproximal shoulder link 306. Alternatively, upon actuation of actuator330, distal shoulder link 308 may be automatically rotated relative toproximal shoulder link 306 until actuator 330 is released, as describedin further detail below with regard to FIG. 56 . For example, actuator330 may be a button or switch operatively coupled to a motor operativelycoupled to distal shoulder link 308 and/or proximal shoulder link 306,such that upon actuation of actuator 330, the associated motor causesdistal shoulder link 308 to rotate relative to proximal shoulder link306. The motor is disposed within the base of robot arm 300, oralternatively, the motor may be disposed adjacent to joint 320, e.g., onshoulder link 305. Accordingly, actuator 330 may be a button or switchthat permits dual actuation, e.g., a first actuation to cause distalshoulder link 308 to rotate in a first direction relative to shoulderlink 306, and a second actuation to cause distal shoulder link 308 torotate in a second direction opposite to the first direction. In someembodiments, the button or switch may be located on a graphical userinterface such as display 110. In addition, in some embodiments, distalshoulder link 308 may be automatically rotated relative to proximalshoulder link 306 by a processor of the co-manipulation robot platform,e.g., during setup, to avoid collisions, as described in further detailbelow.

Robot arm 300 further may include elbow link 310. A proximal end ofelbow link 310 may be rotatably coupled to a distal end of distalshoulder link 308 at elbow joint 322, such that elbow link 310 may berotated relative to distal shoulder link 308 about axis Q4 at elbowjoint 322. Robot arm 300 further may include wrist portion 311, whichmay include proximal wrist link 312 rotatably coupled to the distal endof elbow link 310 at wrist joint 324, middle wrist link 314 rotatablycoupled to proximal wrist link 312 at joint 326, and distal wrist link316 rotatably coupled to middle wrist link 314 at joint 328, as furthershown in FIGS. 4A and 4B. Accordingly, wrist portion 311 may be rotatedrelative to elbow link 310 about axis Q5 at wrist joint 324, middlewrist portion 314 may be rotated relative to proximal wrist link 312about axis Q6 at joint 326, and distal wrist link 316 may be rotatedrelative to middle wrist link 314 about axis Q7 at joint 328. Inaddition, as shown in FIG. 4B, robot arm 300 may include actuator 332,e.g., a lever, button, or switch, operatively coupled to elbow link 310and/or proximal wrist link 312, such that proximal wrist link 312 mayonly be rotated relative to elbow link 310 upon actuation of actuator332. Accordingly, axis Q5 may be a “setup” axis, such proximal wristlink 312 may be rotated and fixed relative to elbow link 310 during asetup stage prior to operating stage where robot arm 300 is used in asurgical procedure. In some preferred embodiments, upon actuation ofactuator 332, proximal wrist link 312 may be manually rotated inpredefined increments relative to elbow link 310, thereby removing thenecessity of having additional motors and/or electronics at the distalregion of robot arm 300. Alternatively, upon actuation of actuator 330,proximal wrist link 312 may be automatically rotated relative to elbowlink 310 until actuator 332 is released.

Referring again to FIG. 3A, robot arm 300 may include a plurality ofmotors, e.g., motors M1, M2, M3, which may all be disposed within thebase of robot arm 300. Each of motors M1, M2, M3 may be operativelycoupled to a respective joint of robot arm 300, e.g., base joint 303,shoulder joint 318, and elbow joint 322, to thereby apply a localizedimpedance at the respective joint. For example, motors M1, M2, M3 mayproduce an impedance at any of base joint 303, shoulder joint 318, andelbow joint 322, respectively, to thereby effectively apply an impedanceat the distal end of robot arm, e.g., at the attachment point with thesurgical instrument, to improve the sensations experienced by theoperator during manipulation of the surgical instrument as well as theactions of the operator during surgical procedures. For example,impedance may be applied to the distal end of robot arm 300, andaccordingly the surgical instrument coupled thereto, to provide asensation of a viscosity, a stiffness, and/or an inertia to the operatormanipulating the surgical instrument. Moreover, applied impedances maysimulate a tissue density or stiffness, communicate surgical boundariesto the operator, and may be used to direct a surgical instrument along adesired path, or otherwise. In some embodiments, the motors may actuatethe respective joints to thereby cause movement of robot arm 300 aboutthe respective joints. Accordingly, axis Q1, axis Q2, and axis Q4 mayeach be a “motorized” axis, such that motors M1, M2, M3 may apply animpedance/torque to base joint 303, shoulder joint 318, and elbow joint322, respectively, to inhibit or actuate rotation about the respectiveaxis. As described in further detail below, motors M1, M2, M3 may becontrolled by a processor of the co-manipulation robot platform. Withthree motorized axes, some implementations of robot arm 300 may applyforce/torque at the distal end of robot arm 300 in three directions tothereby move the surgical instrument coupled to the distal end of robotarm 300 in three degrees of freedom.

Axis Q6 and axis Q7 may be a “passive” axis, such that middle wrist link314 may be rotated relative to proximal wrist link 312 without anyapplied impedance from system 200, and distal wrist link 316 may berotated relative to middle wrist link 314 without any applied impedancefrom system 200. The distal end of distal wrist link 316 may includesurgical instrument coupler interface 400 for removably coupling with asurgical instrument, e.g., via coupler body 500 as shown in FIGS. 4A and4B, which may be removeably coupled to the surgical instrument and tocoupler interface 400, as described in further detail below.Alternatively, wrist portion 311 may include a passive ball joint at theattachment point with the surgical instrument, as described in U.S. Pat.No. 10,582,977, the entire disclosure of which is incorporated herein byreference.

Referring again to FIG. 3A, robot arm 300 further may include aplurality of encoders, e.g., encoders E1–E7, disposed on at least someof the plurality of joints of robot arm 300. For example, encoder E1 formeasuring angulation of between base portion 302 and shoulder portion304 may be disposed on or adjacent to base joint 303 within the base,encoder E2 for measuring angulation of between shoulder portion 304 andproximal shoulder link 306 may be disposed on or adjacent to shoulderjoint 318 within the base, encoder E3 for measuring angulation ofbetween proximal shoulder link 306 and distal shoulder link 308 may bedisposed on or adjacent to joint 320, encoder E4 for measuringangulation of between distal shoulder link 308 and elbow link 310 may bedisposed adjacent to motor M3 operatively coupled to elbow joint 322within the base as transmission of rotational motion at elbow joint 322is achieved via a connection rod extending from the base to elbow joint32, encoder E5 for measuring angulation of between elbow link 310 andproximal wrist link 312 may be disposed on or adjacent to wrist joint324, encoder E6 for measuring angulation of between proximal wrist link312 and middle wrist link 314 may be disposed on or adjacent to joint326, and encoder E7 for measuring angulation of between middle wristlink 314 and distal wrist link 316 may be disposed on or adjacent tojoint 328. Alternatively, encoder E4 may be disposed on or adjacent toelbow joint 322. The encoders may be absolute encoders or otherposition/angulation sensors configured to generate data for accuratelydetermining the position and/or angulation of corresponding links at therespective joint and/or the exact position of the surgical instrumentcoupled to the distal end of robot arm 300. Accordingly, the exactposition of each link, joint, and the distal end of robot 300 may bedetermined based on measurements obtained from the plurality ofencoders. Preferably, a redundant encoder is disposed at each locationalong robot arm 300 where an encoder is placed, to provide more accurateposition data, as well as, to detect a fault condition, as described infurther detail below.

Prior to attachment with a surgical instrument, robot arm 300 may bemanually manipulated by a user, e.g., to position robot arm 300 is adesired position for coupling with the surgical instrument. For example,the user may manually manipulate robot arm 300 via wrist portion 311,actuator 330, and/or actuator 332. Upon actuation of actuator 330, theuser may manually rotate distal shoulder link 308, and upon actuation ofactuator 332, the user may manually manipulate proximal wrist portion312.

In some embodiments, responsive to force applied to robot arm 300, e.g.,at wrist portion 311, wrist joint 324, elbow link 310, etc., by theuser, e.g., in a predetermined amount or pattern, in a given direction,the processor of the co-manipulation robot platform may cause the stagecoupled to base portion 302 of robot arm 300 to move robot arm 300 inthe same direction, e.g., via the vertical and horizontal extenders ofthe stage, until the force applied to robot arm 300 by the user isdetected by the system to drop below a predetermined threshold, e.g.,when the user releases robot arm 300. In some embodiments, the systemmay stop movement of robot arm 300 in the same direction as the forceapplied by the user when the user applies a counter force to robot arm300, e.g., in a direction opposite to the direction of movement of robotarm 300, to facilitate setup of robot arm 300 relative to the patient.

For example, the user may apply a force that exceeds a predeterminedforce threshold on wrist portion 311 in a first direction, which causesthe stages of platform 100 to move robot arm 300 in that same directionuntil the user stops movement of wrist portion 311, e.g., by letting goof robot arm 300 or by applying a counter force to robot arm 300, suchthat the system stops movement of the stages of platform 100.Accordingly, movement of the distal end of the robot arms, e.g., wristportion 311, wrist joint 324, elbow link 310, etc., may serve as aninput for motion generated in particular directions of the robot armsvia the stages coupled thereto. Such automated movement of the stages ofplatform 100 responsive to force applied to the distal end of robot arm300 by the user may be limited to when the system is in a predefinedoperating mode, which may be entered in during setup and/or during asurgical procedure, e.g., upon actuation at GUI 110 and/or via voicecontrol.

Upon attachment to the surgical instrument, robot arm 300 may still bemanipulated manually by the user exerting force, e.g., one or morelinear forces and/or one or more torques, directly to robot arm 300;however, during the laparoscopic procedure, the operator preferablymanipulates robot arm 300 only via the handle of the surgicalinstrument, which applies force/torque to the distal end of the robotarm 300, and accordingly the links and joints of robot arm 300. As theoperator applies a force to the surgical instrument attached to robotarm 300, thereby causing movement of the surgical instrument, robot arm300 will move responsive to the movement of the surgical instrument toprovide the operator the ability to freely move surgical instrumentrelative to the patient. As described in further detail below, robot arm300 may apply an impedance to account for weight of the surgicalinstrument and of robot arm 300 itself, e.g., gravity compensation, asthe operator moves the surgical instrument, thereby making it easier forthe operator to move the instrument despite gravitational forces and/orinertial forces being exerted on the robot arm and/or the surgicalinstrument. As will be understood by a person having ordinary skill inthe art, robot arm 300 may include less or more articulation joints thanis shown in FIG. 3A, as well as a corresponding number of motors andencoders/sensors.

Referring now to FIG. 4C, a close-up view of the coupling mechanism ofcoupler interface 400 and coupler body 500 is provided. Couplerinterface 400 may be coupled to the distal end of distal wrist link 316using any suitable fasteners or connectors, e.g., magnets, screws, pins,clamps, welds, adhesive, rivets, and/or any other suitable faster or anycombination of the foregoing. As shown in FIG. 4C, coupler interface 400may be coupled with the distal end of distal wrist portion 316 usingfastener 410 which may be threaded or have other features that enablefastener 410, and accordingly coupler interface 400 to be selectivelyattached to distal wrist portion 316. Fastener 410 may be coupled withinsert element 408 having an opening therein to receive fastener 410,positioned at or in the distal end of distal wrist portion 316. In someembodiments, fastener 410 may be a pin or may have other features suchas a ball, a latch, or otherwise to permit fastener 410 to selectivelycouple with distal wrist portion 316.

Coupler body 500, which may have opening 514 sized and shaped toslidably and releasably receive the elongated shaft of a surgicalinstrument therethrough, may be removably coupled with coupler interface400. For example, coupler body 500 may be removeably coupled to couplerbody 500 via a magnetic connection, to thereby facilitate efficientattachment and detachment between coupler body 500 and coupler interface400, e.g., by overcoming the magnetic coupling force between couplerbody 500 and coupler interface 400. Accordingly, as shown in FIG. 4C,coupler body 500 may have one or more magnets 506 extending away from asurface of coupler body 500 that, in an assembled state, contacts asurface of coupler interface 400. Alternatively, in embodiments that donot have a coupler interface, magnets 506 may directly contact thedistal end of distal wrist portion 316. Accordingly, coupler interface400 or the distal end of distal wrist portion 316 may have a ferrousbase component configured to receive and magnetically couple withmagnets 506 of coupler body 500 so that coupler body 500 may beremovably coupled with coupler interface 500 and/or the distal end ofdistal wrist portion 316.

FIG. 4D illustrates surgical instrument coupler interface 400. As shownin FIG. 4D, coupler interface 400 may have recessed portion 404 sizedand shaped to receive the complementary geometry of coupler body 500,defined by ridges 402. Accordingly, when the complementary geometry ofcoupler body 500 is received in recessed portion 404 in an assembledstate, rotational movement of coupler body 500 relative to couplerinterface 400 may be limited or otherwise prevented. In addition,coupler interface 400 may have one or more recesses or depressions 406sized and shaped to receive one or more magnets 506 therein. Couplerinterface 400 may have a ferrous base component or magnets withinrecesses 406 to magnetically couple with magnets 506. For example, themagnets within recesses 406 may have a south magnetic pole and magnets506 may have a north magnetic pole, or vice versa. Moreover, thepolarity of the magnets can ensure appropriate coupling orientation.Recesses 406 may be sized and shaped to limit or otherwise preventmovement between coupler body 500 and coupler interface 400 in anydirection that is radial or normal to an axial (e.g., longitudinal)centerline of magnets 506 when coupler body 500 is in an assembled statewith coupler interface 400. As will be understood by a person havingordinary skill in the art, coupler interface 400 may have less or morethan two recesses 406, such that coupler body 500 will have acorresponding amount of magnets.

Referring now to FIGS. 5A and 5B, coupler body 500 is provided. As shownin FIG. 5A, coupler body 500 may have one or more magnets 506 disposedon portion 502 having a geometry complementary to recessed portion 404of coupler interface 400, as described above, to facilitate alignmentbetween coupler body 500 and coupler interface 400. In addition, couplerbody 500 may have one or more grooves 504 sized and shaped to engagewith complementary ridges 402 of coupler interface 400. Grooves 504 andridges 402 may interact to assist with the alignment of coupler body 500with coupler interface 400 by limiting or otherwise preventing movementbetween coupler body 500 and coupler interface 400 in at least twodirections D1 and D2, as shown in FIG. 4C. Accordingly, in an assembledstate, coupler body 500 may be prevented from moving in any axialdirection relative to coupler interface 400.

As shown in FIGS. 5A and 5B, coupler body 500 may have first portion 508and second portion 510. First portion 508 may be coupled with, orintegrally formed with, second portion 510, e.g., via hinge 512, whichmay be a living hinge formed from the same material as first and secondportions 508, 510 and/or integrally formed with first and secondportions 508, 510 so that second portion 510 may be moved or rotatedrelative to first portion 508 to cause opening 514 defined by firstportion 508 and second portion 510 to expand (increase in size) orcontract (decrease in size). First portion 508 and second portion 510may form a clamp that may constrict about the elongated shaft of asurgical instrument that is positioned in opening 514 as screw 516,e.g., a thumb screw, is tightened, to couple the instrument 112 with thecoupler body 141. Accordingly, coupler body 500 may transition between afirst, unsecured/open state or position and a second, secured/closedstate or position.

The diameter of opening 514 may be selected based on the surgicalinstrument to be coupled to coupler body 500. For example, a couplerbody may be selected from a plurality of coupler bodies, each couplerbody having an opening sized and shaped to receive the elongate shaft ofa specific surgical instrument having a predefined elongated shaftdiameter such as a laparoscopic or other surgical instrument includingsurgical instruments used for orthopedic and trauma surgery (OTS), aneedle holder, clamp, scissors, etc. Coupler body 500 may be coupledwith the surgical instrument at any desired axial position on thesurgical instrument.

As shown in FIG. 4C, coupler body 500 may include recess 520 extendingthrough second portion 510 and recess 522 extending through at least aportion of first portion 508. Recess 520 is aligned with recess 522 forreceiving locking portion 518 of screw 516. For example, locking portion518 may have a male threaded surface, and recesses 520, 522 may have afemale threaded surface to engage with locking portion 518. Screw 516may be loosened by hand to open or expand opening 514 so that thesurgical instrument may be removed, repositioned, rotated, and/or slid,etc. Once coupler body 500 is coupled with the surgical instrument,e.g., via screw 516, coupler body 500 and the surgical instrument thatis coupled with the coupler body 500 may be removeably coupled withcoupler interface 400, via magnets 506.

Opening 514 may be defined by a first semi-circular cutout in firstportion 508 and a second semi-circular cutout in the second portion 510of coupler body 500, to thereby engage with the circular outer surfaceof the elongate shaft of a surgical instrument. Opening 514 may include,e.g., rubber pads, sheets, bumps, O-rings, projections, or othercomponents or features configured to contact and grip the outer surfaceof the elongated shaft of the surgical instrument. For example, therubber material may be a silicone rubber or any other suitable type ofrubber. Accordingly, once coupler body 500 is coupled with the surgicalinstrument, e.g., by securing screw 516, the surgical instrument may beat least inhibited or otherwise prevented from moving axially, e.g., thedirection along the longitudinal axis of the surgical instrument, or, insome embodiments, moving axially and rotationally, relative to couplerbody 500 in the secured state. Preferably, the surgical instrumentcoupled with coupler body 500 may be freely rotated by an operatorrelative to coupler body 500, while axial movement of the surgicalinstrument relative to coupler body 500 is inhibited or otherwiseprevented in the secured state. For example, the frictional forcebetween the outer surface of the elongated shaft of the surgicalinstrument and the inner surface of coupler body 500 defining opening514 may be selected such that rotation of the surgical instrumentrelative to coupler body 500 requires less force that axial movement ofthe surgical instrument relative to coupler body 500 in the securedstate. Accordingly, coupler 500 may be configured to account fordiametric variations and surface variations (including variations in acoefficient of friction of the surface) of the surgical instruments.

In some embodiments, the surgical instrument may be moved in an axialdirection relative to coupler body 500 upon the application of at leasta threshold force on the surgical instrument relative to coupler body500, or upon actuation of a release or a state change of coupler body500. For example, such actuation may be achieved by, e.g., pressing abutton, loosening a locking screw such as locking screw 516 or otherconnector, moving a dial, or otherwise changing coupler body 500 and/orcoupler interface 400 from a second, secured state to a first, unsecuredstate. Accordingly, the surgical instrument may be axially repositionedrelative to coupler body 500 by loosening screw 516 or otherhand-operated fastener or fastening mechanism such as a clamp in couplerbody 500, repositioning the surgical instrument in the desired axialposition, and re-tightening screw 516 or other hand-operated fastener orfastening mechanism. Coupler body 500 may be disposable, oralternatively, may be sterilizeable such that it may sterilized betweensurgical procedures.

As described above, the diameter of the opening of the coupler body maybe selected based on the surgical instrument to be coupled to thecoupler body. Most commonly used laparoscopic surgical instruments havea predefined, known elongated shaft diameter, and thus the numerouscoupler bodies may be provided, each having an opening sized and shapedto receive and engage with a specific surgical instrument. For example,FIG. 6A illustrates coupler body 600 having opening 614 sized and shapedto receive a 5 mm diameter surgical instrument, e.g., retractor device12. Coupler body 600 may be constructed similar to coupler body 500. Forexample, coupler body 600 may include first portion 608 coupled tosecond portion 610 via hinge portion 612, and recesses 620, 622 forsecurely receiving locking portion 618 of screw 616. As shown in FIG.6B, coupler body 600 may receive elongated shaft 12 a of retractor 12through opening 614, e.g., from the operating end of retractor 12, suchthat coupler body 600 may be slid over elongated shaft 12 a untilcoupler body 600 engages with proximal portion 12 b of retractor 12, asshown in FIG. 6C. Preferably, coupler body 600 is coupled to retractor12 when coupler body 600 contacts proximal portion 12 b as this pointalong retractor 12 is fixed, thereby providing a consistent point ofreference for calculating force measurements, as described in furtherdetail below. Accordingly, when coupler body 600 is in the desiredlocation along the elongated shaft of retractor 12, e.g., adjacent toproximal portion 12 b, screw 616 may be coupled to coupler body 600 tosecure coupler body 600 to retractor 12. As described above, couplerbody 600 is secured to retractor 12 such that rotational movement ofretractor 12 relative to coupler body 600 is permitted, while axialmovement of retractor 12 relative to coupler body 600 is constrained,e.g., the force required to move retractor 12 relative to coupler body600 is much higher than the force required to rotate retractor 12relative to coupler body 600.

FIG. 7A illustrates coupler body 700 having opening 714 sized and shapedto receive a 10 mm diameter surgical instrument, e.g., laparoscopedevice 10. Coupler body 700 may be constructed similar to coupler body600. For example, coupler body 700 may include first portion 708 coupledto second portion 710 via hinge portion 712, and recesses 720, 722 forsecurely receiving locking portion 718 of screw 716. As shown in FIG.7B, coupler body 700 may receive elongated shaft 10 a of laparoscopedevice 10 through opening 714, e.g., from the operating end oflaparoscope 10, such that coupler body 700 may be slid over elongatedshaft 10 a until coupler body 700 engages with proximal portion 10 b oflaparoscope 10, as shown in FIG. 7C. Preferably, coupler body 700 iscoupled to laparoscope 10 when coupler body 700 contacts proximalportion 10 b as this point along laparoscope 10 is fixed, therebyproviding a consistent point of reference for calculating forcemeasurements, as described in further detail below. Accordingly, whencoupler body 700 is in the desired location along the elongated shaft oflaparoscope 10, e.g., adjacent to proximal portion 10 b, screw 716 maybe coupled to coupler body 700 to secure coupler body 700 to laparoscope10. As described above, coupler body 700 is secured to laparoscope 10such that rotational movement of laparoscope 10 relative to coupler body700 is permitted, while axial movement of laparoscope 10 relative tocoupler body 700 is constrained, e.g., the force required to movelaparoscope 10 relative to coupler body 700 is much higher than theforce required to rotate laparoscope 10 relative to coupler body 700.

With the appropriate sized coupler body coupled to the selected surgicalinstrument, the coupler body may be removeably coupled to couplerinterface 400 of robot arm 300. Coupler body 500 and coupler interface400 may be configured for single-handed coupling, such that an operatormay couple coupler body 500, and accordingly the surgical instrumentcoupled thereto, to coupler interface 400 of robot arm 300 using asingle hand. Preferably, a surgical drape may be pinched or clampedbetween the coupler body and coupler interface 400, and draped overrobot arm 300 to maintain sterility of the surgical space and preventcontact with non-sterile components of robot arm 300. Accordingly, thesterile drape may pass continuously (e.g., without a hole, a slit, orany other type of opening) between the coupler body and the couplerinterface such that the coupler body is on a first side of the steriledrape and the coupler interface, robot arm 300, and/or other componentsof system 200 are on the other side of the sterile drape. In someembodiments, the coupler body may be integrated with the surgical drape.Additionally or alternatively, the surgical drape may include an adapterintegrated therewith, such that coupler body 500 may be coupled tocoupler interface 400 via the adapter, e.g., the adapter may bepositioned between coupler body 500 and coupler interface 400.

Referring now to FIGS. 8A and 8B, robot arm 300 may be positioned in asurgical drape-ready configuration. As shown in FIG. 8A, robot arm 300may be extended such that wrist portion 311, elbow link 310, andshoulder link 305 extend away from shoulder portion 304 of the base topermit a surgical/sterile drape to be draped over each component ofrobot arm 300. Moreover, as shown in FIG. 8B, when there are two robotarms, e.g., robot arm 300 a and robot arm 300 b, robot arm 300 a androbot arm 300 b may be angled away from each other, e.g., by rotatingshoulder portion 304 a relative to base portion 302 a of robot arm 300 aand by rotating shoulder portion 304 b relative to base portion 302 b ofrobot arm 300 b, such that wrist portion 311 a, elbow link 310 a, andshoulder link 305 a extend away from wrist portion 311 b, elbow link 310b, and shoulder link 305 b. This configuration permits efficient andaccessible draping of the respective robot arms with a surgical/steriledrape. Moreover, in the extended position, the robot arms may be outsidethe virtual haptic boundary, such that the robot arms are in the hapticmode and a high level of impedance is applied to the robot arms therebymaking movement of the robot arms more viscous, which makes it easierfor the operator to drape the robot arms, yet provide movement theretoif necessary. For example, FIG. 9A illustrates a single robot arm 300draped with sterile drape 800, and FIG. 9B illustrates robot arms 300 a,300 b draped with sterile drapes 800 a, 800 b, respectively.

Sterile drape 800 may be completely closed at an end portion thereof. Insome embodiment, sterile drape 800 may have an opening (that canoptionally have a sterile seal or interface) in a distal portion thereofthat a portion of robot arm 300, coupler interface 400, coupler body500, and/or the surgical instrument may pass through. Drapes having asealed end portion without any openings, and being sealed along a lengththereof may provide a better sterile barrier for system 200.Accordingly, all of robot arm 300 may be located inside sterile drape800 and/or be fully enclosed within sterile drape 800, except at anopening at a proximal end of sterile drape 800, e.g., near the base ofrobot arm 300). In some embodiments, coupler body 500 and couplerinterface 400 may have electrical connectors to produce an electronicconnection between robot arm 300 and the surgical instrument.Accordingly, the electrical signals may be transmitted through steriledrape 800. Alternatively, sterile drape 800 may include an opening suchthat electrical wires or other components may pass through the openingto provide a wired communication channel to electrical components thatmay include, e.g., memory chips for calibration, radiofrequency probesfor ablation, cameras, and other electronic components. The surgicalinstrument and the coupler body may instead be passive or non-electronicsuch that no electrical wires need pass through sterile drape 800.

Referring now to FIGS. 10A to 10D, rotation of distal shoulder link 308relative to proximal shoulder link 306 of shoulder link 305 is provided.As described above, axis Q3 may be a “setup” axis, such that distalshoulder link 308 may be rotated relative to proximal shoulder link 306upon actuation of actuator 330 during a setup stage of robot arm 300,e.g., prior to operation of robot arm 300 in a surgical procedure. Asshown in FIG. 10A, shoulder portion 304 optionally may be initiallyrotated relative to base portion 302 to a desired position, therebycausing rotation of all the link distal to proximal shoulder link 306,which is coupled to shoulder portion 304, to rotate relative to baseportion 302 and provide ample space for rotation of robot arm 300 aboutjoint 320. Moreover, as shown in FIG. 10 , wrist portion 311 may be atleast partially extended away from base portion 302 so as to not collidewith any components of robot arm 300 upon rotation of robot arm 300about joint 320. As shown in FIG. 10B, actuator 330 must be actuated topermit rotation of distal shoulder link 308 relative to proximalshoulder link 306 at joint 320. FIG. 10C illustrates robot arm 300 in adesirable location for a specific laparoscopic procedure upon rotationof distal shoulder link 308 relative to proximal shoulder link 306. FIG.10D illustrates robot arm 300 a in the desirable location upon rotationof distal shoulder link 308 a relative to proximal shoulder link 306 a,relative to robot arm 300 b. In some embodiments, joint 320 may beoperatively coupled to a motor, such that distal shoulder link 308 maybe automatically rotated relative to proximal shoulder link 306, asdescribed in further detail with regard to FIG. 56 .

Referring now to FIGS. 11A and 11B, an exemplary co-manipulation robotsurgical system having an optical scanner is provided. As shown in FIG.11A, the system may be constructed similar to system 200 of FIG. 2 ,having a plurality of robot arms, e.g., robot arm 300 a and robot arm300 b. As described above, although only two robot arms are shown inFIG. 11A, less or more robot arms may be used in conjunction withoptical scanner 1100. In addition, the system may include one or moreoptical scanners 1100, e.g., a LiDAR scanner or other suitable opticalscanning device such as an RGBD camera or sensor, RGB camera withmachine learning, a time-of-flight depth camera, structured light,multiple projection cameras, a stereo camera, ultrasound sensors, laserscanner, other type of coordinate measuring area scanner, or anycombination of the foregoing. For example, the LiDAR camera/scanner maybe capable of recording both color (RGB) and the Depth (D) of thesurgical field, and may include, for example, an Intel RealSense LiDARCamera L515 or an Intel RealSense Depth Camera D435i (made available byIntel, Santa Clara, California) or other LiDAR or depth cameras havingsimilar or suitable specifications including, without limitation, any ofthe following specifications: (i) range: 25 cm to 500 cm; depthaccuracy: 5 mm or approximately 5 mm; depth field of view: 70 × 55 orapproximately 70 × 55 (degrees); depth output resolution: 1024 × 768pixels or approximately 1024 × 768 pixels; depth/RGB frame rate: 30frames per second; RGB frame resolution: 1920 × 1080; and/or RGB fieldof view: 70 × 43 degrees or approximately 70 × 43 degrees. The LiDARscanner or optical scanner further may include both a ¼-20 UNC thread or2x M3 thread mounting points. As will be understood by a person havingordinary skill in the art, optical scanner 1100 may be used in otherco-manipulation robot surgical systems described herein, e.g., system200, or any variations thereof.

As shown in FIG. 11A, the platform supporting robot arms 300 a, 300 bmay support optical scanner 1100, and any other electronics, wiring, orother components of the system, such that optical scanner 1100 ismounted in a fixed location relative to the other objects in thesurgical space, and the position and orientation of optical scanner 1100is known or may be determined with respect to the global coordinatesystem of the system, and accordingly, the robot arms. This allows alldata streams to be transformed into a single coordinate system fordevelopment purposes. For example, optical scanner 1100 may be supportedon a rod or shaft, e.g., rod 1102, which may have an adjustable heightor otherwise be adjustable in any direction, e.g., up/down, left/right,toward/away from the patient, to allow optical scanner 1100 to gain anoptimum field-of-view or position relative to the other components ofthe system, for example, robot arms 300 a, 300 b, the surgicalinstruments attached thereto, the surgeon, and/or surgical assistant.Moreover, telemetry data captured by optical scanner 1100, e.g.,indicative of the movements of the surgeon’s hands, other body parts,the patient bed, the trocar, the surgical instruments, and othercomponents of the system, may be recorded to provide a rich and detaileddataset describing the precise movements and forces applied by thesurgeon throughout the procedure.

For example, the data obtained may be used to optimize the proceduresperformed by the system including, e.g., automatic servoing (i.e.,moving) of one or more portions of robot arm 300. By tracking thetendency of the surgeon to keep the tools in a particular region ofinterest and/or the tendency of the surgeon to avoid moving the toolsinto a particular region of interest, the system may optimize theautomatic servoing algorithm to provide more stability in the particularregion of interest. In addition, the data obtained may be used tooptimize the procedures performed by the system including, e.g.,automatic re-centering of the field of view of the optical scanningdevices of the system. For example, if the system detects that thesurgeon has moved or predicts that the surgeon might move out of thefield of view, the system may cause the robot arm supporting the opticalscanning device, e.g., a laparoscope, to automatically adjust thelaparoscope to track the desired location of the image as the surgeonperforms the desired procedure. This behavior may be surgeon-specificand may require an understanding of a particular surgeon’s preferencefor an operating region of interest. Thus, the system may control therobot arms pursuant to specific operating requirements and/orpreferences of a particular surgeon. Moreover, if the system detectsthat the robot arms are in an extended position for a period of timeexceeding a predetermined threshold, the system may cause the stagescoupled to the base portions of the robot arms to move the robot arms ina manner to ease extension of the robot arms, and thereby provideadditional range for extension of the robot arms by the user.

Referring now to FIG. 11C, another exemplary co-manipulation robotsurgical system having a plurality of optical sensors is provided. Asshown in FIG. 11C, system 200 has a plurality of robot arms, e.g., robotarm 300 a and robot arm 300 b, supported by platform 100 having aplurality of wheels for providing mobility to platform 100. As describedabove, the plurality of wheels may each include a braking mechanismwhich may be actuated to be engaged and prevent movement of platform100. For example, the braking mechanism may be operatively coupled to acontroller of system 200. Moreover, system 200 may include a pluralityof optical sensors, e.g., optical scanners 1100 a, 1100 b, and 1100 c,disposed on platform 100. For example, optical scanner 1100 a may bedisposed on top of platform 100, as described above with regard tooptical scanner 1100 of FIG. 11A, and optical scanners 1100 b and 1100 cmay be disposed on the sides of platform 100. Additionally oralternatively, one or more optical scanners may be disposed underneathplatform 100. Optical scanners 1100 a, 1100 b, and 1100 c are configuredto capture depth data. For example, optical scanners 1100 a, 1100 b, and1100 c may be, e.g., a depth camera, a stereo RGB camera, a LIDARdevice, and/or an electromagnetic, capacitive, or infrared proximitysensor, etc.

The depth data generated by the plurality of optical sensors may be usedby the controller of system 200 to generate a virtual map, e.g., a“bird’s eye view”, of the area surrounding platform 100, e.g., withinthe operating room, in real-time. For example, the virtual map mayillustrate the operating room from a top perspective. Moreover, as shownin FIG. 11C, the virtual map may include graphical representations ofplatform 100 (including robot arms 300 a, 300 b), as well as one or moreobjects, e.g., patient table PT, and/or one or more persons, e.g.,operator O, person P1, and person P2, within the area surroundingplatform 100. Specifically, the virtual map may graphically illustratethe proximity between platform 100 and the one or more objects/persons,e.g., as platform 100 is being moved through the operating room byoperator O. The controller may cause display 110 to display the virtualmap, such that operator O can view the virtual map on display 110 inreal-time as operator O moves platform 100 through the operating room.Accordingly, operator O may see objects and/or persons in the areasurrounding platform 100 that operator O could not otherwise see withtheir own eyes, e.g., due to platform 100 and/or robot arms 300 a, 300 bobstructing the view of operator O, and avoid collisions betweenplatform 100 and/or robot arms 300 a, 300 b with the objects/persons inthe operating room. Moreover, the controller may cause display 110 todisplay an alert, e.g., a visual or audible alert, when the virtual mapindicates that platform 100 and/or robot arms 300 a, 300 b areapproaching or within a predetermined distance from the one or moreobjects/persons within the operating room.

In some embodiments, the controller may only cause display 110 todisplay the virtual map while platform 100 is being moved within theoperating room. For example, platform 100 may include one or moreactuators, e.g., a button, lever, or handlebar, that may be operativelycoupled to the braking mechanism of the wheels of platform 100, suchthat upon actuation of the actuator, the braking mechanism is disengagedsuch that mobility of platform 100 is permitted. Accordingly, when theactuator is not actuated, the braking mechanism is engaged such thatmobility of platform 100 is prevented. Thus, upon actuation of theactuator, the controller may automatically cause display 110 to displaythe virtual map, such that operator O can view the area surroundingplatform 100 before, during, or after movement of platform 100 while thebraking mechanism is disengaged. Once the actuator is released, suchthat the braking mechanism is reengaged, display 110 may stop displayingthe virtual map. In some embodiments, when the virtual map indicatesthat platform 100 and/or robot arms 300 a, 300 b are approaching orwithin the predetermined distance from the one or more objects/personswithin the operating room, the controller may override actuation of theactuator by the operator and reengage the braking mechanism to therebyprevent further movement of platform 100. Accordingly, the actuator mayneed to be released and re-actuated by the operator to disengage thebraking mechanism and permit further movement of platform 100.

Moreover, the system may process color and/or depth data obtained fromoptical scanners 1100 a, 1100 b, and/or 1100 c to identify objectswithin the operating room, e.g., the patient bed or the trocar, as wellas the planes associated with the identified objects. With knowledge ofthe location platform 100 and robot arms 300 a, 300 b relative to theidentified objects, the system may cause the stages coupled to the baseportions of robot arms 300 a, 300 b to automatically move (or stopmovement of) robot arms 300 a, 300 b to avoid collision with theidentified objects during setup, e.g., when robot arms 300 a, 300 bapproaches a predetermined distance threshold relative to the identifiedobjects. In addition, the system may generate and emit, e.g., an audiblealert indicative of the proximity of the stages of platform 100 and/orrobot arms 300 a, 300 b relative to the identified objects. For example,the audible alert may change in amplitude and/or frequency as thedistance between the stages of platform 100 and/or robot arms 300 a, 300b and the identified objects decreases, as perceived by the system basedon the depth data.

Additionally, with knowledge of the location platform 100 and robot arms300 a, 300 b relative to the patient and the trocar, in combination withknowledge of where robot arms 300 a, 300 b are positioned relative tothe patient and the trocar for a given surgical procedure, the systemmay automatically position robot arms 300 a, 300 b in a setupconfiguration relative to the patient and the trocar for the givensurgical procedure. For example, the system may automatically positionthe distal end of the robot arm adjacent to the trocar, and further mayarrange the robot arm in a predetermined configuration, e.g., via themotorized joints of the robot arm, that is preferred for the givensurgical procedure.

In addition, with knowledge of the location platform 100 and robot arms300 a, 300 b relative to the trocar, if the system detects that theposition of the patient bed, and accordingly the trocar, is changing,e.g., via adjustment by a user, the system may automatically adjust thearrangement of the robot arm to accommodate the movement of the patientbed and maintain relative position between the distal end of the robotarm and the trocar. In some embodiments, upon detection of movement ofthe patient bed, the system may automatically move the robot arm toretract the surgical instrument coupled thereto within the trocar, priorto automatically adjusting the arrangement of the robot arm to maintainrelative position between the distal end of the robot arm and thetrocar, such that the distal end of the surgical instrument ispositioned within the trocar and away from anatomical structures withinthe patient.

FIG. 12 shows the system having optical scanner 1100 in operation duringa laparoscopic procedure. As shown in FIG. 12 , an optional additionaloptical scanner, e.g., camera 1200, may be utilized to provide anadditional point of view, e.g., redundant measurement of the movementsof the instruments held by the robot arms, and/or provide a video streamof the surgical scene, e.g., via streaming, for monitoring and analysis.As shown in FIG. 12 , the system may include two robot arms, e.g., robotarms 300 a, 300 b, such that robot arm 300 a holds laparoscope 10 in afixed position relative to the patient, while the surgeon operates andmanipulates retractor 12, which is coupled to the distal end of robotarm 300 b. Moreover, during the surgical procedure, robot arms 300 a,300 b may be draped with sterile drapes 800 a, 800 b, respectively. Asdescribed above, the surgeon may freely manipulate retractor 12 whileretractor 12 is coupled to robot arm 300 b, thereby causing movement ofrobot arm 300 b due to movement of retractor 12 by the surgeon, andwhile robot arm 300 b accounts for weight of retractor 12 and robot arm300 b. During the surgical procedure, optical scanner 1100 may be usedto monitor an identity, position, orientation, and/or movement of thesurgical instrument coupled to robot arm 300 a, e.g., laparoscope 10,and an identity, position, orientation, and/or movement of the surgicalinstrument coupled to robot arm 300 b, e.g., retractor 12, as well as ifeither surgical instrument is detached from the respective robot arm,either intentionally or unintentionally. Moreover, optical scanner 1100may be used to monitor an identity, position, orientation, and/ormovement/displacement of any of trocars Tr to ensure proper alignment ofthe robot arms and/or surgical instruments relative to the respectivetrocars. The system may be used in a surgical procedure having one, two,three, four, or more trocars, depending on the surgical procedureintended to be performed by the system.

FIGS. 13A and 13B illustrate exemplary data produced by optical scanner1100. For example, FIG. 13A illustrates image data captured by opticalscanner 1100, and FIG. 13B illustrates a depth map of at least someobjects within the surgical space generated from the data captured byoptical scanner 1100. Specifically, optical scanner 1100 may create adepth map, e.g., point clouds, where each pixel’s value is related tothe distance from optical scanner 1100. For example, the differencebetween pixels for a first object (such as a first surgical instrument)and a second object (for example, a trocar) will enable the system tocalculate the distance between the surgical instrument and the trocar.Moreover, the difference between pixels for a first object (such as afirst surgical instrument) at a first point in time and the first objectat a second point in time will enable the system to calculate whetherthe first object has moved, the trajectory of movement, the speed ofmovement, and/or other parameters associated with the changing positionof the first object.

As shown in FIGS. 13A and 13B, surgeon S is manipulating surgical toolsand/or the draped robot arm (DA) and the undraped robot arm (UA) thatare positioned relative to insufflated abdomen (A). As described above,the data streams from the robot arms, the camera feed from thelaparoscope, the data acquired from optical scanner 1100, as well asdata optionally captured from one or more imaging devices disposed on astructure adjacent to the robot arms, the walls, ceiling, or otherstructures within the operating room, may be recorded, stored, and usedindividually or in combination to understand and control the surgicalsystem and procedures of the surgical system. The foregoing components,devices, and combinations thereof are collectively referred to herein asoptical scanners or optical scanning devices.

For example, the system may measure and record any of the followingwithin the coordinate space of the system: motion of the handheldsurgical instruments manipulated by the surgeon (attached to or apartfrom a robot arm); the presence/absence of other surgical staff (e.g.,scrub nurse, circulating nurse, anesthesiologist, etc.); the height andangular orientation of the surgical table; patient position and volumeon the surgical table; presence/absence of the drape on the patient;presence/absence of trocar ports, and if present, their position andorientation; gestures made by the surgical staff; tasks being performedby the surgical staff; interaction of the surgical staff with thesystem; surgical instrument identification; attachment or detachment“action” of surgical instruments to the system; position and orientationtracking of specific features of the surgical instruments relative tothe system (e.g., camera head, coupler, fiducial marker(s), etc.);measurement of motion profiles or specific features in the scene thatallow for the phase of the surgery to be identified; position,orientation, identity, and/or movement of any other instruments,features, and/or components of the system or being used by the surgicalteam.

The system may combine measurements and/or other data described abovewith any other telemetry data from the system and/or video data from thelaparoscope to provide a comprehensive dataset with which to improve theoverall usability, functionality, and safety of the co-manipulationrobot-assisted surgical systems described herein. For example, as thesystem is being setup to start a procedure, optical scanner 1100 maydetect the height and orientation of the surgical table. Thisinformation may allow the system to automatically configure the degreesof freedom of platform 100 supporting robot arms 300 to the desired orcorrect positions relative to the surgical table. Specifically, opticalscanner 1100 may be used to ensure that the height of platform 100 isoptimally positioned to ensure that robot arms 300 overlap with theintended surgical workspace. In addition, as described above, the systemmay automatically reconfigure the degrees of freedom of platform 100 aswell as the arrangement of robot arms 300 responsive to movement of thesurgical table, and accordingly the trocar(s), to maintain relativeposition between the distal end of the robot arms and the trocar(s).

Moreover, based on the data obtained by optical scanner 1100, the systemmay alert the surgical staff of a potential collision (either duringsetup or intra-operatively) between the system and other pieces ofcapital equipment in the operating room, e.g., the surgical table, alaparoscopic tower, camera booms, etc., as well as with a member of thesurgical staff, e.g., an inadvertent bump by the staff member. Thesystem may use this information to recommend a repositioning of platform100 and/or other components of the system, the surgical table, and/orpatient, and/or prevent the robot arm from switching to theco-manipulation mode as a result of the force applied to the robot armby the collision with the staff member, even if the force exceeds thepredetermined force threshold of the robot arm.

In addition, the data obtained from optical scanner 1100 may be used tomonitor the progress of setup for a surgical procedure and may becombined with the known state of the system to inform remote hospitalstaff (e.g., the surgeon) of the overall readiness to start theprocedure. Such progress steps may include: (i) patient on table; (ii)patient draped; (iii) sterile instruments available; (iv) robot armdraped; (v) trocar ports inserted; and (vi) confirmation thatinstruments (e.g., a laparoscope and retractor) are attached to therobotic arms of system. For example, the data obtained from opticalscanner 1100 may include detected gestures indicative of the systemstate (e.g., system is draped), readiness to start the procedure, etc.,and further may be used to prepare the system for the attachment ordetachment of a surgical instrument.

In addition, optical scanner 1100 may identify the specific surgeoncarrying out the procedure, such that the system may use the surgeon’sidentity to load a system profile associated with the particular surgeoninto the system. The system profile may include information related to asurgeon’s operating parameter and/or preferences, a surgeon’s patientlist having parameters for each patient, the desired or requiredalgorithm sensitivity for the surgeon, the degree of freedom positioningof the support platform, etc. Examples of algorithm sensitivities thatmay be surgeon-specific include: adapting/adjusting the force requiredto transition from passive mode to co-manipulation mode (e.g., from lowforce to high force), adapting/adjusting the viscosity felt by thesurgeon when co-manipulating the robot arm (e.g., from low viscosity tohigh viscosity), etc. Moreover, the surgeon’s preferences may includepreferred arrangements of robot arm 300, e.g., the positioning of thelinks and joints of robot arm 300 relative to the patient, with regardto specific surgical instruments, e.g., the preferred arrangement may bedifferent between a laparoscope and a retractor.

In some embodiments, the surgeon’s preferences may be learned based ondata from past procedures and/or sensors collecting information aboutcurrent procedure including a surgeon’s current pose, a surgeon’sheight, a surgeon’s hand preference, and other similar factors. Forexample, the system may record when a user interacts with the system andalso record what the user does with the system, such that the datasetmay allow for surgeon preferences to be “learned” and updated over time.This learning may be done either via traditional algorithmic methods(i.e., trends over time, averaging, optical flow, etc.) or via machinelearning approaches (classification, discrimination, neural networks,reinforcement learning, etc.). FIG. 24 illustrates data flow 2400 forupdating the system configurations based on learned behaviors of theuser. As shown in FIG. 24 , the system may be connected to an onlinedatabase that may store a surgeon profile and each of a plurality ofpossible data sources, which may include optical sensors, encoders,and/or other sensors, and/or a database of manually entered user input.The data sources may be associated with a given surgeon, their preferredrobot arm arrangement and operating parameters, and each procedureperformed with the system, which may allow the recording and analysis ofthe system configuration and how it changes from procedure to procedure,and within the procedure. In the case of machine learning, theco-manipulation capability of the system may be leveraged such that theuser’s actions may be used to annotate the data to create a trainingdataset.

Regarding the degree of freedom positioning, a height of a surgicaltable is typically adjusted to accommodate the height of the surgeon insome operating rooms. Thus, by detecting the surgeon and loading thesurgeon’s specific profile, the system may position the platform at aheight that is suitable for the respective surgeon to accommodate thepreferred height of the surgical table. In addition, the horizontaltranslation of a robot arm may depend on the size of the patient. Thus,by accessing the patient list, the system may adjust the position of thearm based on the patient’s body mass index (“BMI”). For example, for apatient with a high BMI, the system may move the robot arm away from theoperating table and, for a patient with a low BMI, the system may movethe robot arm closer to the operating table. Accordingly, the systempermits the surgical team to fine-tune the position of the robot armrelative to the patient as necessary. The system further may beconfigured to access a hospital medical record database to access theprocedure type and any other medical data available (e.g., CT scanimages, x-ray images, MRI images, and/or other patient specificinformation), which may be used to inform positioning of the trocarports, and the position and orientation of platform 100 relative to thepatient.

Based on the data captured by optical scanner 1100, the system maygenerate a virtual model of the pieces of capital equipment and/or otherobjects in an operating room that are within a range of movement of therobot arms in the same co-ordinate space as the robot arms and surgicalinstruments coupled thereto, such that the virtual model may be storedand monitor, e.g., to detect potential collisions. Additionally, thesystem may track the position and orientation of each virtual model, andthe objects within the virtual models as the objects move relative toeach other, such that the system may alert the user if the proximity of(i.e., spacing between) any of the virtual models or objects falls belowa predefined threshold, e.g., within 50 mm, 75 mm, from 30 mm or less to100 mm, or more. In some embodiments, the distance threshold may bebased off the Euclidean distance between the closest points on twovirtual models, the normal distance between two surfaces of the virtualmodels, etc. Moreover, the system may stop or inhibit (e.g., prevent)further movement of a robot arm, e.g., freeze the robot arm, if theproximity of any of the virtual models or objects, e.g., a robot armreaches or falls below the predefined threshold relative to alaparoscopic tower, or the surface of the surgical table, or otherobjects within the surgical space. In addition, the system may freezethe robot arm if the system detects that the proximity between anobject, e.g., capital equipment or a member of the surgical staff otherthan the surgeon, moving toward a respective robot arm reaches or fallsbelow the predefined threshold, to thereby prevent the inadvertentmovement of the robot arm that may otherwise result from such acollision or inadvertent force, e.g., an inadvertent bump from a memberof the staff or another piece of capital equipment, etc.

In addition, based on the data captured by optical scanners 1100 a, 1100b, 1100 c, the system may generate a virtual map with graphicalrepresentations of objects and/or persons that are within a predefinedarea surrounding the platform and robot arms in an operating room in thesame co-ordinate space as the platform and robot arms, such that thevirtual map may be stored and displayed to a user, e.g., to detectpotential collisions while the user moves the platform throughout theoperating room. Additionally, the system may track the position andorientation of the graphical representations within the virtual map,such that the system may alert the user if the proximity between any ofthe objects and/or persons from the platform and/or robot arms fallswithin a predetermined threshold, e.g., within 50 mm, 75 mm, from 30 mmor less to 100 mm, or more.

Moreover, based on the data captured by optical scanner 1100, the systemmay track the motion of the handheld surgical instruments that aredirectly and independently controlled by the surgeon, that are notcoupled with the robot arm. For example, the optical scanner 1100 maytrack a clearly defined feature of the instrument, a fiducial markerattached to the instrument or to the gloves (e.g., the sterile gloves)of the surgeon, the coupler between the robot arm and the instrument, adistal tip of the instrument, and/or any other defined location on theinstrument. For example, fiducial markers may include Manus virtualreality gloves (made available by Manus, The Netherlands) or otherwearables, and/or the OptiTrack systems (made available by NaturalPoint,Corvallis, Oregon). The following are examples of uses and purposes ofthe motion data: (i) closing a control loop between a handheldinstrument and the robot arm holding the camera, thus allowing thesurgeon to servo (i.e., move) the camera by “pointing” with a handheldinstrument; (ii) tracking information that may be used independently orin combination with other data streams to identify the phase of thesurgical procedure; (iii) to identify the dominant hand of the surgeon;(iv) to monitor metrics associated with the experience of the surgeon;(v) to identify which tools the surgeon is using and when to change themfor other tools; and/or (vi) tracking of the skin surface of thepatient, as well as the number, position and orientation of the trocarports. This data and information also may be used and computed by thesystem as part of the co-manipulation control paradigm. By measuring thetrue position and orientation of the trocar ports, the system may beprovided an additional safety check to ensure that the system levelcomputations are correct, e.g., to ensure that the actual motion of therobot arms or instrument matches a commanded motion of the robot arms orinstrument in robotic assist mode.

Based on the data captured by optical scanner 1100, the system furthermay track the which instrument is being used in a respective port, howoften instruments are swapped between ports, which ports have manuallyheld instruments versus instruments coupled to the robot arm, to monitorand determine if additional trocar ports are added, if the system isholding the instruments in place while the patient or surgical table ismoving (in which case, the system may change the operational mode of therobot arms to a passive mode and accommodate the movement byrepositioning robot arm 300 and/or platform 100), and/or otherconditions or parameters of the operating room or the system. Theknowledge of the position and orientation of the skin surface and trocarports relative to the robot arms may facilitate the implementation of“virtual boundaries” as described in further detail below.

Moreover, based on the data obtained by optical scanner 1100, e.g.,tracked movements of the distal end of a laparoscope coupled to robotarm 300, in addition to image data captured by the laparoscope, thesystem may identify the type of laparoscope coupled to robot arm 300.For example, laparoscopes commonly used during laparoscopic proceduresinclude flat-tipped laparoscopes and angled-tipped laparoscopes, e.g., alaparoscope having a 30 degree angled tip. The system may determinewhich laparoscope type is currently coupled to robot arm 300 bycomparing the image data obtained by optical scanner 1100 of apredefined pattern of movement of the laparoscope, e.g., moving thedistal end of the laparoscope in a circular pattern in a planeperpendicular to the longitudinal axis of the laparoscope, with theimage data obtained by the laparoscope as the laparoscope is being movedin the predefined pattern of movement. For example, for a flat-tippedlaparoscope, the image data captured by the laparoscope as the distalend of the laparoscope is moved in a circular pattern in the planeperpendicular to the longitudinal axis of the laparoscope should movealong a circular planar path, e.g., there will be no change in depth ofthe field of view of the laparoscope; whereas, for an angled-tippedlaparoscope, the image data captured by the laparoscope as the distalend of the laparoscope is moved in a circular pattern in the planeperpendicular to the longitudinal axis of the laparoscope will observe achange of depth of the field of view of the laparoscope.

In addition, the system may calibrate a new robot arm when a currentrobot arm is replaced, e.g., during a surgical procedure, based on thedata obtained by optical scanner 1100, with or without utilizing atracker at the distal end of the new robot arm, to ensure the system isaccurately aware of the kinematics of the new robot arm. Specifically,the system may calibrate optical scanner 1100 to platform 100, calibratethe new robot arm with respect to the base portion of the new robot arm,and calibrate the new robot arm with respect to platform 100 when thenew robot arm is coupled to platform 100. For example, based on thetelemetry data obtained by optical scanner 1100, the system may comparethe actual real-time movements of the new robot arm as captured byoptical scanner 1100 to the movements expected based on commands sent tothe new robot arm by the system, e.g., to execute a preprogrammedroutine intended to move the new robot arm in specific positions, andgenerate a degree of error indicative of a deviation between the actualreal-time movements of the new robot arm and the expected movements ofthe robot arm based on the preprogrammed routine. The system further mayexecute an optimization algorithm to reduce or eliminate the degree oferror between the actual real-time movements and the expected movements,e.g., until the degree of error falls below a predetermined threshold.This calibration process may occur when the system is in a predefinedcalibration mode, or alternatively, in real-time during a surgicalprocedure after the new robot arm is coupled to platform 100.

Based on the data obtained by optical scanner 1100, e.g., knowledge ofthe position and orientation of the surgical bed relative to platform100 and/or the trocar port(s), the system may automatically positionrobot arms 300 in a preferred configuration relative to the patientduring setup via the motorized joints of robot arm 300 and/or the stagesof platform 100 based on the surgical procedure to be performed, e.g.,responsive to actuation by a user via GUI 110, while avoiding collisionsbetween the stages of platform 300, robot arm 300, and objects in theoperating room such as the surgical bed. Accordingly, the system maystore preset robot arm configurations, e.g., based on stored surgeonpreferences, for various surgical procedures.

Referring now to FIG. 14 , components that may be included inco-manipulation robot platform 1400 are described. Platform 1400 mayinclude one or more processors 1402, communication circuitry 1404, powersupply 1406, user interface 1408, and/or memory 1410. One or moreelectrical components and/or circuits may perform some of or all theroles of the various components described herein. Although describedseparately, it is to be appreciated that electrical components need notbe separate structural elements. For example, platform 1400 andcommunication circuitry 1404 may be embodied in a single chip. Inaddition, while platform 1400 is described as having memory 1410, amemory chip(s) may be separately provided.

Platform 1400 may contain memory and/or be coupled, via one or morebuses, to read information from, or write information to, memory. Memory1410 may include processor cache, including a multi-level hierarchicalcache in which different levels have different capacities and accessspeeds. The memory also may include random access memory (RAM), othervolatile storage devices, or non-volatile storage devices. Memory 1410may be RAM, ROM, Flash, other volatile storage devices or non-volatilestorage devices, or other known memory, or some combination thereof, andpreferably includes storage in which data may be selectively saved. Forexample, the storage devices can include, for example, hard drives,optical discs, flash memory, and Zip drives. Programmable instructionsmay be stored on memory 1410 to execute algorithms for, e.g.,calculating desired forces to be applied along robot arm 300 and/or thesurgical instrument coupled thereto and applying impedances atrespective joints of robot arm 300 to effect the desired forces.

Platform 1400 may incorporate processor 1402, which may consist of oneor more processors and may be a general purpose processor, a digitalsignal processor (DSP), an application specific integrated circuit(ASIC), a field programmable gate array (FPGA) or other programmablelogic device, discrete gate or transistor logic, discrete hardwarecomponents, or any suitable combination thereof designed to perform thefunctions described herein. Platform 1400 also may be implemented as acombination of computing devices, e.g., a combination of a DSP and amicroprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration.

Platform 1400, in conjunction with firmware/software stored in thememory may execute an operating system (e.g., operating system 1446),such as, for example, Windows, Mac OS, QNX, Unix or Solaris 5.10.Platform 1400 also executes software applications stored in the memory.For example, the software may be programs in any suitable programminglanguage known to those skilled in the art, including, for example, C++,PHP, or Java.

Communication circuitry 1404 may include circuitry that allows platform1400 to communicate with an image capture devices such as opticalscanner and/or endoscope. Communication circuitry 1404 may be configuredfor wired and/or wireless communication over a network such as theInternet, a telephone network, a Bluetooth network, and/or a WiFinetwork using techniques known in the art. Communication circuitry 1404may be a communication chip known in the art such as a Bluetooth chipand/or a WiFi chip. Communication circuitry 1404 permits platform 1400to transfer information, such as force measurements on the body wall atthe trocar insertion point locally and/or to a remote location such as aserver.

Power supply 1406 may supply alternating current or direct current. Indirect current embodiments, power supply may include a suitable batterysuch as a replaceable battery or rechargeable battery and apparatus mayinclude circuitry for charging the rechargeable battery, and adetachable power cord. Power supply 1406 may be a port to allow platform1400 to be plugged into a conventional wall socket, e.g., via a cordwith an AC to DC power converter and/or a USB port, for poweringcomponents within platform 1400. Power supply 1406 may be operativelycoupled to an emergency switch, such that upon actuation of theemergency switch, power stops being supplied to the components withinplatform 1400 including, for example, the braking mechanism disposed onat least some joints of the plurality of joints of robot arm 300. Forexample, the braking mechanisms may require power to disengage, suchthat without power supplied to the braking mechanisms, the brakingmechanisms engage to prevent movement of robot arm 300 without power.

User interface 1408 may be used to receive inputs from, and/or provideoutputs to, a user. For example, user interface 1408 may include atouchscreen, display, switches, dials, lights, etc. Accordingly, userinterface 1408 may display information such as selected surgicalinstrument identity and force measurements observed during operation ofrobot arm 300. Moreover, user interface 1408 may receive user inputincluding adjustments to the predetermined amount of movement at thehandle of the surgical instrument or the predetermined dwell time periodto cause the robot arm to automatically switch to the passive mode, thepredetermined threshold of force applied at the handle of the surgicalinstrument to cause the robot arm to automatically switch to theco-manipulation mode, a position of the predefined haptic barrier, anidentity of the surgical instrument coupled to the distal end of therobot arm, a vertical height of the robot arm, a horizontal position ofthe robot arm, etc., such that platform 1400 may adjust theinformation/parameters accordingly. In some embodiments, user interface1408 is not present on platform 1400, but is instead provided on aremote, external computing device communicatively connected to platform1400 via communication circuitry 1404.

Memory 1410, which is one example of a non-transitory computer-readablemedium, may be used to store operating system (OS) 1446, surgicalinstrument identification module 1412, surgical instrument calibrationmodule 1414, encoder interface module 1416, robot arm positiondetermination module 1418, trocar position detection module 1420, forcedetection module 1422, impedance calculation module 1424, motorinterface module 1426, optical scanner interface module 1428, gesturedetection module 1430, passive mode determination module 1432,co-manipulation mode determination module 1434, haptic modedetermination module 1436, robotic assist mode determination module1438, fault detection module 1440, indicator interface module 1442, andfatigue detection module 1444. The modules are provided in the form ofcomputer-executable instructions/algorithms that may be executed byprocessor 1402 for performing various operations in accordance with thedisclosure.

For example, during a procedure, the system may continuously run thealgorithms described herein based on the data collected by the system.That data may be collected and/or recorded using any of the componentsand methods disclosed herein, including, e.g., from sensors/encoderswithin the robots, from optical scanning devices in communication withthe other components of the robotic system, and/or from manual inputs byan operator of the system. Accordingly, the algorithms, the data, andthe configuration of the system may enable the user to co-manipulate therobot arms with minimal impact and influence from the weight of therobot arms and/or surgical instruments coupled thereto, force ofgravity, and other forces that traditional robot arms fail to compensatefor. Some of the parameters of the algorithms described herein maycontrol an aspect of the behavior of the system including, e.g.,robustness of detected features, sensitivity to false positives, robotcontrol gains, number of features to track, dead zone radius, etc.

Surgical instrument identification module 1412 may be executed byprocessor 1402 for identifying the surgical instrument coupled to eachof the robot arms, and loading the appropriate calibration file into thecontroller system. For example, the calibration file for each surgicalinstrument may be stored in a database accessible by surgical instrumentidentification module 1412, and may include information associated withthe surgical instrument such as, e.g., instrument type, weight, centerof mass, length, instrument shaft diameter, etc. Accordingly, when theappropriate calibration file is loaded, and the associated surgicalinstrument is coupled to robot arm 300, the system will automaticallyaccount for the mass of the surgical instrument, e.g., compensate forgravity on the surgical instrument, when the surgical instrument isattached to robot arm 300 based on the data in the calibration file,such that robot arm 300 may hold the surgical instrument in positionafter the surgical instrument is coupled to the robot arm and theoperator lets go of the surgical instrument. For example, surgicalinstrument identification module 1412 may identify the surgicalinstrument based on user input via user interface 1408, e.g., theoperator may select the surgical instrument from a database of surgicalinstruments stored in memory 1410.

In some embodiments, surgical instrument identification module 1412 mayautomatically identify the surgical instrument coupled with the roboticarm via the coupler body and the coupler interface using, e.g., an RFIDtransmitter chip and reader or receiver (e.g., placing an RFID stickeror transmitter on the surgical instrument that may transmit informationabout the surgical instrument to a receiver of the system), an nearfield communication (“NFC”) device such as a near field magneticinduction communication device, a barcode and scanner or other opticaldevice, a magnet based communication system, reed switches, a Bluetoothtransmitter, the weight of the instrument and/or data gathered from theoptical scanner and a lookup table, and/or any other features ormechanisms described herein or suitable for identification of thesurgical instrument. As described above, the coupler body may beselected based on the size and shape of the lumen extending therethroughto accommodate and engage with a surgical instrument having a knownelongated shaft diameter. Accordingly, surgical instrumentidentification module 1412 may automatically identify the surgicalinstrument based on the coupler body that is coupled to the surgicalinstrument via the magnetic connection between the coupler body and thecoupler interface.

In some embodiments, e.g., when using coupling mechanism 4400 describedbelow with regard to FIGS. 44A-50B, surgical instrument identificationmodule 1412 may identify the surgical instrument, e.g., the type ofsurgical instrument, based on data obtained from a sensor, e.g., a Halleffect sensor within distal wrist link 316, indicative of a magneticfield strength induced by a magnet displaced responsive to coupling ofthe surgical instrument to the coupler body coupled to the couplerinterface. Based on the strength of the detected magnetic field, thesystem may determine whether the coupler body is coupled to the couplerinterface, whether a surgical instrument is coupled to the coupler body,and the diameter of the shaft of the surgical instrument, e.g., a 5 mmor 10 mm surgical instrument, as described in further detail below.

Surgical instrument identification module 1412 further may cause thedistal end of robot arm 300, when coupled to a laparoscope, to move in apredefined pattern, e.g., in a circular motion in a plane perpendicularto the longitudinal axis of the laparoscope, and compare the image datacaptured by the laparoscope as the distal end of the laparoscope ismoved in the predefined circular pattern with image data obtained byoptical scanner 1100 of the laparoscope as the laparoscope is beingmoved in the predefined pattern to identify whether the laparoscope is aflat-tipped laparoscope or an angled-tip laparoscope, as describedabove.

In some embodiments, surgical instrument identification module 1412 mayidentify the surgical instrument, e.g., the type of surgical instrument,based on data obtained by optical scanner 1100 via optical scannerinterface module 1428 described in further detail below. For example,the data may include measurement data associated with the specificinstrument, such that surgical instrument identification module 1412 maycompare such data with information contained within the database toidentify the instrument and load the appropriate calibration file intothe controller system. Similarly, surgical instrument identificationmodule 1412 may detect if the instrument is removed and return thecalibration parameters to a default configuration.

Surgical instrument calibration module 1414 may be executed by processor1402 for calibration a surgical instrument, e.g., a surgical instrumentthat does not currently have an associated calibration file in thedatabase stored in memory 1410. Accordingly, surgical instrumentcalibration module 1414 may calculate measurements and specifications ofa surgical instrument when it is coupled to robot arm 300 and the systemis in calibration mode, as described in further detail below with regardto FIG. 16 , based on force measurements of robot arm 300 applied by thesurgical instrument via force detection module 1422. For example,surgical instrument calibration module 1414 may generate a calibrationfile for the surgical instrument including information such asinstrument type, weight, center of mass, length, instrument shaftdiameter, a viscosity parameter of the surgical instrument, etc. Atleast some of the surgical instrument information in the calibrationfile may be provided by user input via user interface 1408, e.g., theinstrument type, or may be detected by optical scanner interface module1428, e.g., the instrument type, the center of mass of the instrument,the instrument length, and the instrument diameter.

If surgical instrument calibration module 1414 determines thatre-calibration results are consistently different from theconfigurations already loaded into the system, surgical instrumentcalibration module 1414 may replace existing information or add to itslist of known tools without any user inputs and load them automatically.Surgical instrument calibration module 1414 may determine that thecalibration factors are not adequate to compensate for the force ofgravity if, e.g., when a surgical instrument is coupled with the robotarm, the robot arm moves due only to forces of gravity acting on therobot arm and/or the surgical instrument, which may be done when thesurgical instrument is positioned completely outside of the patient’sbody. Moreover, surgical instrument calibration module 1414 mayautomatically update or adjust the calibration factors (e.g., the forcesapplied to the joints of the robot arm) if it determines that thecalibration factors are not adequate to compensate for the force ofgravity. Thus, surgical instrument calibration module 1414 may updatethe calibration factors for the particular surgical instrument and storethe updated calibration factors for the particular surgical instrumentin the associated calibration file for future use.

Encoder interface module 1416 may be executed by processor 1402 forreceiving and processing angulation measurement data from the pluralityof encoders of robot arm 300, e.g., encoders E1–E7, in real time. Forexample, encoder interface module 1416 may calculate the change inangulation over time of the links of robot arm 300 rotatably coupled toa given joint associated with the encoder. As described above, thesystem may include redundant encoders at each joint of robot arm 300, tothereby ensure safe operation of robot arm 300. Moreover, additionalencoders may be disposed on platform 100 to measure angulation/positionof each robot arm relative to platform 100, e.g., the vertical andhorizontal position of the robot arms relative to platform 100.Accordingly, an encoder may be disposed on platform 100 to measuremovement of the robot arms along the vertical axis of platform 100 andanother encoder may be disposed on platform 100 to measure movement ofthe robot arms along the horizontal axis of platform 100.

Robot arm position determination module 1418 may be executed byprocessor 1402 for determining the position of robot arm 300 and thesurgical instrument attached thereto, if any, in 3D space in real timebased on the angulation measurement data generated by encoder interfacemodule 1416. For example, robot arm position determination module 1418may determine the position of various links and joints of robot arm 300as well as positions along the surgical instrument coupled to robot arm300. Based on the position data of robot arm 300 and/or the surgicalinstrument, robot arm position determination module 1418 may calculatethe velocity and/or acceleration of movement of robot arm 300 and thesurgical instrument attached thereto in real time. For example, bydetermining the individual velocities of various joints of robot arm300, e.g., via the encoder associated with each joint of the variousjoints, robot arm position determination module 1418 may determine theresultant velocity of the distal end of robot arm 300, which may be usedby passive mode determination module 1432 to determine whether movementof the distal end of robot arm 300 is within a predetermined thresholdfor purposes of transitioning system 200 to passive mode, as describedin further detail below.

Trocar position detection module 1420 may be executed by processor 1402for determining the position and/or orientation of one or more trocarport inserted within the patient. The position and/or orientation of atrocar port may be derived based on data obtained from, e.g., inertialmeasurement units and/or accelerometers, optical scanners,electromechanical tracking instruments, linear encoders, the sensors anddata as described above. For example, the position of the trocar portson the patient may be determined using a laser pointing system that maybe mounted on one or more of the components of the system, e.g., wristportion 311 of the robot arm, and may be controlled by the system topoint to the optimal or determined position on the patient’s body toinsert the trocar. Moreover, upon insertion of the surgical instrumentthat is attached to robot arm 300 through a trocar, virtual lines maycontinuously be established along the longitudinal axis of the surgicalinstrument, the alignment/orientation of which may be automaticallydetermined upon attachment of the surgical instrument to couplerinterface 400 via the coupler body via the magnetic connection asdescribed above, in real time as the surgical instrument moves about thetrocar point. Moreover, when the surgical instrument is inserted withinthe trocar port, it will be pointing toward the trocar point, andaccordingly, distal wrist link 316 will also point toward the trocarpoint, the angle of which may be measured by an encoder associatedtherewith. Accordingly, the trocar point may be calculated as theintersection of the plurality of virtual lines continuously establishedalong the longitudinal axis of the surgical instrument. In this manner,the calculated trocar point will remained fixed relative to the patientas the surgical instrument is maneuvered about the trocar port, e.g.,rotated or moved in or out of the patient. In addition, the orientationof the trocar port and its position relative to robot arm 300 may bedetermined based on image data received from one or more opticalscanners, e.g., a LiDAR camera and/or an RGBD camera.

Based on the known position and/or orientation of a trocar port inaddition to the known position of the distal end of robot arm 300 fromrobot arm position determination module 1418, the system may maintainthe position of the distal end of robot arm 300 relative to the trocarpoint as robot arm 300 moves, e.g., via vertical or horizontaladjustment thereof by platform 100, or as the patient table height isadjusted, thereby causing the height of the patient’s abdomen to move,thereby keeping the surgical instrument within the patient’s body andcoupled to robot arm 300 steady during these external movements. Toachieve this, the known position of the distal end of robot arm 300 fromrobot arm position determination module 1418 is calculated in the globalframe of the system by adding position of platform 100 to the kinematicscalculations (e.g., the “forward kinematics” of robot arm 300 in thecontext of serial chain robotic manipulators).

With the position of the distal end of robot arm 300 known globally, thesystem may hold that position steady by applying appropriate forces torobot arm 300 during the external movements that minimize the errorbetween its current and desired positions. Accordingly, for example,when a surgical instrument coupled to the distal end of robot arm 300 isinserted through a trocar port such that the tip of the instrument isinside of the patient, and a user adjusts the height of the patienttable, the system may apply forces/torques to robot arm 300 toreconfigure robot arm 300 and/or cause movement of the stages ofplatform 100 to maintain the relative position between the distal end ofrobot arm 300, and accordingly the surgical instrument, and the trocarport. In some embodiments, the system may cause the distal end of robotarm 300 to retract slightly such that the tip of the surgical instrumentis positioned within the trocar port and out of contact with anatomicalstructures within the patient’s body prior to reconfiguring robot arm300 to maintain the relative position between the surgical instrumentand the trocar port.

Force detection module 1422 may be executed by processor 1402 fordetecting forces applied on robot arm 300, e.g., at the joints or linksof robot arm 300 or along the surgical instrument, as well as applied onthe trocar, e.g., body wall forces. For example, force detection module1422 may receive motor current measurements in real time at each motor,e.g., M1, M2, M3, disposed within the base of robot arm 300, which areeach operatively coupled to a joint of robot arm 300, e.g., base joint303, shoulder joint 318, elbow joint 322, wrist joint 332. The motorcurrent measurements are indicative of the amount of force applied tothe associated joint. Accordingly, the force applied to each joint ofrobot arm 300 as well as to the surgical instrument attached thereto maybe calculated based on the motor current measurements and the positiondata generated by robot arm position determination module 1418 and/ortrocar position detection module 1420.

Due to the passive axes at the distal end of robot arm 300, the forceapplied by the instrument coupled with the robot arm on the trocar mayremain generally consistent throughout the workspace of the robot arm.The force on the trocar may be affected by the interaction of the distaltip of the instrument with tissue within the body. For example, if atissue retractor advanced through the trocar is engaged with (e.g.,grasping) bodily tissue or another object inside the body, the forceexerted on the end of the instrument from the bodily tissue or otherobject may cause a change in the force applied to the trocar. In someaspects, the force on the trocar may be a function of how much weight isbeing lifted by the instrument being used.

Impedance calculation module 1424 may be executed by processor 1402 fordetermining the amount of impedance/torque needed to be applied torespective joints of robot arm 300 to achieve the desired effect, e.g.,holding robot arm 300 in a static position in the passive mode,permitting robot arm 300 to move freely while compensating for gravityof robot arm and the surgical instrument attached thereto in theco-manipulation mode, applying increased impedance to robot arm 300 whenrobot arm 300 and/or the surgical instrument attached thereto is withina predefined virtual haptic barrier in the haptic mode, etc.

For example, impedance calculation module 1424 may determine the amountof force required by robot arm 300 to achieve the desired effect basedon position data of robot arm 300 generated by robot arm positiondetermination module 1418 and the position data of the trocar generatedby trocar position detection module 1420. For example, by determiningthe position of the distal end of robot arm 300, as well as the point ofentry of the surgical instrument into the patient, e.g., the trocarposition, and with knowledge of one or more instrument parameters, e.g.,mass and center of mass of the surgical instrument stored by surgicalinstrument calibration module 1414, impedance calculation module 1424may calculate the amount of force required to compensate for gravity ofthe surgical instrument (compensation force), as described in furtherdetail below with regard to FIG. 18A. Accordingly, the amount ofcompensation force required to compensate for the gravity of thesurgical instrument may be converted to torque to be applied at thejoints of robot arm 300, e.g., by the motors operatively coupled to thejoints of robot arm 300, as indicated by the motor current measurements.

Moreover, by determining the position of the distal end of robot arm300, and accordingly, a change in position of the distal end of robotarm 300 over time, for example, due to an external force applied to thedistal end of robot arm 300, e.g., by tissue held by the operating endof the surgical instrument, and with knowledge of one or more instrumentparameters, e.g., mass, center of mass, and length of the surgicalinstrument stored by surgical instrument calibration module 1414,impedance calculation module 1424 may calculate the amount of forcerequired to maintain the surgical instrument in a static position (holdforce), as described in further detail below with regard to FIG. 18B.Accordingly, the amount of hold force required to resist the change inposition of the distal end of robot arm 300, in addition to the amountof compensation force required to compensate for the gravity of thesurgical instrument, may be converted to torque to be applied at thejoints of robot arm 300 to maintain robot arm 300 in a static position,e.g., by the motors operatively coupled to the joints of robot arm 300,as indicated by the motor current measurements. In addition, impedancecalculation module 1424 and/or force detection module 1422 may calculatethe amount of force applied by the surgical instrument to the patient atthe point of entry, e.g., at the trocar, as well as the amount of forceapplied to the operating end of the surgical instrument, e.g., thegrasper end of a surgical instrument, based on the compensation force,the hold force, one or more parameters of the surgical instrument suchas the mass, center of mass, and length of the surgical instrument, andthe distance from the center of mass to the point of entry.

Additionally or alternatively, by determining the forces applied onrobot arm 300 via force detection module 1422, as well as theposition/velocity/acceleration of the distal end of robot arm 300 in 3Dspace via robot arm position determination module 1418, the desiredforce/impedance to be applied to robot arm 300 to compensate for theapplied forces may be calculated, e.g., for gravity compensation or tohold robot arm 300 in a static position in the passive mode.Accordingly, the desired force may be converted to torque to be appliedat the joints of robot arm 300, e.g., by the motors operatively coupledto the joints of robot arm 300. For example, the robot Jacobian may beused for this purpose. Jacobian is a matrix that is computer at eachgiven post of the robot arm, and relates the velocities at the joints tothe velocity at the distal end of robot arm 300:

V = J * q_(dot)

Here, V is the velocity vector at the distal end of robot arm 300, J isits Jacobian matrix, and q_(dot) is its joint velocities expressed invector form. Using the energy principle, and assuming negligible massesfor the links of robot arm 300 and negligible friction/dampening, thepower of the system may be determined by multiplying its force andvelocity:

$\begin{matrix}{F \cdot V = \tau \cdot q_{dot}} \\{= >} \\{F \cdot \left( {J \ast q_{dot}} \right) = \tau \cdot q_{dot}}\end{matrix}$

Here, F is the generalized force vector at the distal end of robot 300.Further, vector manipulation results in:

$\begin{matrix}{\left( {J^{t} \ast F} \right) \cdot q_{dot} = \tau \cdot q_{dot}} \\{= >} \\{\tau = J^{t} \ast F}\end{matrix}$

Here, t denotes the transpose of the matrix, such that the forces at thedistal end of robot arm 300 may be converted to torques to be applied atthe joints using the Jacobian matrix.

Motor interface module 1426 may be executed by processor 1402 forreceiving motor current readings at each motor, e.g., M1, M2. M3,disposed within the base of robot arm 300, and for actuating therespective motors, e.g., by applying a predetermined impedance toachieved the desired outcome as described herein and/or to cause thejoints operatively coupled to the respective motors to move, such as inthe robotic assist mode. In some embodiments, when joint 320 isoperatively coupled to a motor, e.g., M4, such that distal shoulder link308 may be automatically rotated relative to proximal shoulder link 306,as described in further detail with regard to FIG. 56 , motor interfacemodule 1426 may actuate M4 to cause rotation of distal shoulder link 308relative to proximal shoulder link 306.

Optical scanner interface module 1428 may be executed by processor 1402for receiving depth data obtained by optical scanner 1100 and processingthe depth data to detect, e.g., predefined conditions therein. Moreover,optical scanner interface module 1428 may generate depth maps indicativeof the received depth data, which may be displayed to the operator,e.g., via a monitor. For example, optical scanner interface module 1428may map the location of the trocar ports in 3D space, such that themapping of trocar ports may be communicated to the operator, e.g., viadisplay or user interface 1408. Based on depth data and/or color datareceived from optical scanner 1100, optical scanner interface module1428 may determine the relative distances between, e.g., the stages ofplatform 100, robot arm 300, any surgical instruments attached thereto,and objects/persons in the operating room such as the surgical table,drapes, etc.

Optical scanner interface module 1428 further may receive image datafrom additional optical scanning devices as defined herein, includingfor example, an endoscope operatively coupled to the system. Moreover,optical scanner interface module 1428 may receive depth data obtained byoptical scanners 1100 a, 1100 b, 1100 c coupled to platform 100 andprocess the depth data to generate a virtual map of the area surroundingplatform 100, as described above with regarding to FIG. 11C, which maybe displayed to the operator via a monitor, e.g., display 110. Forexample, optical scanner interface module 1428 may generate graphicalrepresentations of system 200 including platform 100 and robot arms 300a, 300 b, and any objects and/or persons within the area surroundingplatform 100 for display in the virtual map.

Gesture detection module 1430 may be executed by processor 1402 fordetecting predefined gestural patterns as user input, and executing anaction associated with the user input. The predefined gestural patternsmay include, for example, movement of a surgical instrument (whether ornot attached to robot arm 300), movement of robot arm 300 or othercomponents of the system, e.g., foot pedal, buttons, etc., and/ormovement of the operator in a predefined pattern. For example, movementof the surgical instrument back and forth in a first direction (e.g..,left/right, up/down, forward/backward, in a circle) may be associatedwith a first user input requiring a first action by the system and/orback and forth in a second direction (e.g.., left/right, up/down,forward/backward, in a circle) that is different than the firstdirection may be associated with a second user input requiring a secondaction by the system. Similarly, pressing the foot pedal or a buttonoperatively coupled with the system in a predefined manner may beassociated with a third user input requiring a third action by thesystem, and movement of the operator’s head back and forth or up anddown repeatedly may be associated with a fourth user input requiring afourth action by the system. Various predefined gestural patternsassociated with different components or operators of the system may beredundant such that the associated user input may be the same fordifferent gestural patterns. The predefined gestural patterns may bedetected by, e.g., an optical scanning device such as a laparoscope oroptical scanner 1100 via optical scanner interface module 1428 ordirectly by force applied to robot arm 300 via force detection module1422 or other components of the system.

Actions responsive to user input associated with predefined gesturalpatterns may include, for example, enabling tool tracking to servo(i.e., move) the laparoscope based on the motion of a handheld tool;engaging the brakes on (e.g., preventing further movement of) the robotarm; engaging a software lock on the robot arm; dynamically changing thelength of time that the robot arm takes to transition between statesfrom a default setting; loading a virtual menu overlay on the video feedwhereby a surgical instrument in the field of view of the laparoscopefunctions as a pointer to trigger further actions available from thevirtual menu; and/or identifying which member of the surgical staff istouching the robot arm, if any. This information may be used to ensurethat the system does not move if the surgeon is not touching the robotarm, e.g., to avoid the scenario where an external force is acting onthe robot arm (e.g., a light cable or other wire being pulled across therobot arm) and the system perceives the force to be intentional from thesurgeon. The same information may be used to detect the gaze directionof the surgeon, e.g., whether the surgeon is looking at the video feedor somewhere else in the room, such that the system may freeze the robotarm if the surgeon’s gaze is not in the direction it should be.Additionally, the system may reposition a field of view of a camerabased on, for example, the direction a surgeon is facing or based on theobjects that the surgeon appears to be looking at, based on the datafrom the optical scanner 1100.

As described above, responsive to detection of a predefined gesturalpattern by the user, e.g., a predefined pattern of movement of thedistal tip of the surgical instrument within the field of view of thelaparoscope, gesture detection module 1430 may cause a virtual menu tooverlay on the video feed, such that the surgical instrument within thefield of view of the laparoscope functions as a pointer, as shown inFIG. 52 . Moreover, gesture detection module 1430 may detect furtherpredefined patterns of movement of the distal end of the surgicalinstrument, e.g., two quick movements in the same direction or acircular movement over a select area of the virtual menu, which may beinterpreted as a selection actuation, e.g., a click on the virtual menu.For example, as shown in FIG. 52 , the virtual menu overlay on the videofeed may include menu options in the corners of the video feed, e.g.,“hot corners”, such as: turning on/off assisted scope mode where thesystem automatically moves the robot arm coupled to a laparoscope tofollow the surgical instrument and/or zoom in or out to change the fieldof view of the laparoscope; adjusting the holding force of robot armcoupled to a retractor, e.g., the amount of force that may be applied tothe distal tip of the surgical instrument before the system transitionsfrom passive mode to co-manipulation mode; turning on/off audio; andturning on/off haptic feedback. As will be understood by a person havingordinary skill in the art, more or less menu options may be provided viathe virtual menu. In some embodiments, initiation of the display of thevirtual menu overlay on the video feed may be triggered by, e.g.,actuation of an external actuator such as a foot pedal, a predefinedpattern of force applied to the robot arm such double tapping wristportion 311 and/or the surgical instrument coupled to the robot arm asdetected by encoders at the distal end of the robot arm, voiceactivation, wireless buttons, hot buttons, etc.

In some embodiments, the operator may actively switch the system to acommand mode, e.g., via user interface 1408, where particular movementsor gestures of the robot arm, surgical instrument, operator, orotherwise as described herein are monitored by gesture detection module1430 to determine if they are consistent with a predefined gesturalpattern associated with a predefined user input.

Passive mode determination module 1432 may be executed by processor 1402for analyzing the operating characteristics of robot arm 300 todetermine whether to switch the operational mode of robot arm 300 to thepassive mode where the system applies impedance to the joints of robotarm 300 via motor interface module 1426 in an amount sufficient tomaintain robot arm 300, and accordingly a surgical instrument attachedthereto, if any, in a static position, thereby compensating for mass ofrobot arm 300 and the surgical instrument, and any other external forcesacting of robot arm 300 and/or the surgical instrument. If robot arm 300is moved slightly while in the passive mode, but not with enough forceto switch out of the passive mode, the system may adjust the amount ofimpedance applied the robot arm 300 to maintain the static position, andcontinue this process until robot arm 300 is held in a static position.For example, passive mode determination module 1432 may determine toswitch the operational mode of robot arm 300 to the passive mode ifmovement of the robot arm due to movement at the handle of the surgicalinstrument as determined by force detection module 1422 is less than apredetermined amount, e.g., no more than 1 to 5 mm, for at least apredetermined dwell time period associated with robot arm 300. Thepredetermined dwell time period refers to the length of time that robotarm 300 and/or the surgical instrument attached thereto, if any, areheld in a static position. For example, the predetermined dwell time mayrange between, e.g., 0.1 to 3 seconds or more, and may be adjusted bythe operator. FIG. 19 illustrates a table or exemplary values of thethreshold dwell times for a range of sample instrument types.

In some embodiments, passive mode determination module 1432 maydetermine to switch the operational mode of robot arm 300 to the passivemode if movement of the distal end of the robot arm due to movement atthe handle of the surgical instrument as determined by force detectionmodule 1422 has a velocity that is less than a predetermined dwellvelocity/speed. For example, if passive mode determination module 1432determines that the distal end of the robot arm 300 and/or the surgicalinstrument attached thereto, if any, moves at a speed that is lower thanthe predetermined dwell speed during an entire predetermined dwellperiod, then passive mode determination module 1432 may switch theoperational mode of robot arm 300 to the passive mode. FIG. 19illustrates a table or exemplary values of the threshold dwell speedsfor a range of sample instrument types. For example, for surgicalinstruments such as scopes and tissue manipulation devices, thethreshold dwell speeds may be, e.g., 3-5 mm/second, and for surgicalinstruments such as suturing instruments, needle drivers, high forceinstruments, staplers, and clip appliers, the threshold dwell speeds maybe, e.g., 1-2 mm/second. In some embodiments, passive mode determinationmodule 1432 may determine to switch the operational mode of robot arm300 to the passive mode based on the identity of the surgical instrumentupon attachment of the surgical instrument to robot arm 300 and/orresponsive detachment of the surgical instrument from robot arm 300.

Co-manipulation mode determination module 1434 may be executed byprocessor 1402 for analyzing the operating characteristics of robot arm300 to determine whether to switch the operational mode of robot arm 300to the co-manipulation mode where robot arm 300 is permitted to befreely moveable responsive to movement at the handle of the surgicalinstrument for performing laparoscopic surgery using the surgicalinstrument, while the system applies an impedance to robot arm 300 viamotor interface module 1426 in an amount sufficient to account for massof the surgical instrument and robot arm 300. Moreover, the impedanceapplied to robot arm 300 may provide a predetermined level of viscosityperceivable by the operator. FIG. 19 illustrates a table or exemplaryvalues of viscosity levels for a range of sample instrument types. Insome embodiments, the viscosity level may be a function of the speedthat the surgical instrument is being moved and the distance of the tipof the instrument from the trocar point. For example, co-manipulationmode determination module 1434 may determine to switch the operationalmode of robot arm 300 to the co-manipulation mode if force applied atrobot arm 300 due to force applied at the handle of the surgicalinstrument exceeds a predetermined threshold associated with robot arm300 (e.g., a “breakaway force”). The predefined force threshold may be,e.g., at least 7 Newtons, approximately 7 Newtons, at least 7 Newtons,4-15 Newtons, 4-10 Newtons. The predefined force threshold may bedependent on the type of surgical instrument that is being used and/orwhether there is an external force being applied to the surgicalinstrument.

FIG. 19 illustrates a table or exemplary values of the predefined forcethresholds for a range of sample instrument types. As shown in FIG. 19 ,the predefined force thresholds may reflect the typical external tissueforces that may be exerted on the surgical instrument. In someembodiments, predefined force threshold may be increased if a force isexerted on the surgical instrument by tissue or an organ or otherwise,depending on the direction of the breakaway force. For example, if thebreakaway force is in the same direction as the force exerted on thesurgical instrument from the tissue or organ, the predefined forcethreshold may be increased by an amount equal to or commensurate withthe force exerted on the surgical instrument from the tissue or organ.In some embodiments, the predefined force threshold for a respectiverobot arm be adjusted based on a patient’s body mass index (“BMI”). Forexample, a patient with a higher BMI may have a heavier liver that wouldlikely exert a greater force on the instrument. Accordingly, thepredefined force threshold may selected to be higher for the patientswith a higher BMI. Accordingly, the operation may actuate a “high forcemode,” e.g., via user interface 1408, where predefined force thresholdis increased to accommodate for engaging with heavier tissue or organs.For example, the predefined force threshold may be selectively increasedby 20-100% or more.

Moreover, the force exerted by the user on the surgical instrument andany external tissue forces applied to the surgical instrument may bedirectionally dependent. For example, if the force exerted by the useron the surgical instrument is in the same direction as an externaltissue force applied to the surgical instrument, the two forces may beadditive such that the amount of force exerted by the user on thesurgical instrument needed to overcome the predefined force thresholdmay be reduced by the magnitude of the external tissue force such that alower force than the predefined force threshold would be required toexit the passive mode and enter the co-manipulation mode. On the otherhand, if the force exerted by the user on the surgical instrument is ina direction opposite to an external tissue force applied to the surgicalinstrument, than the necessary amount of force exerted by the user onthe surgical instrument needed to overcome the predefined forcethreshold may be increased by the magnitude of the external tissue forcesuch that a higher force than the predefined force threshold would berequired to exit the passive mode and enter the co-manipulation mode.

In addition, if the force exerted by the user on the surgical instrumentis in a direction that is perpendicular to an external tissue forceapplied to the surgical instrument, than the necessary amount of forceexerted by the user on the surgical instrument needed to overcome thepredefined force threshold may not be affected by the magnitude of theexternal tissue force such that the necessary force exerted by the useron the surgical instrument needed to exit the passive mode and enter theco-manipulation mode will equal the predefined force threshold. Forother directions, the force vectors of the applied forces may be addedto or offset by the force vectors of the external tissue forces toovercome predefined force threshold values for the system or theparticular surgical instrument that is coupled with the robot arm,depending on the direction of the external tissue force, if any, and theforce applied by the user. In some embodiments, co-manipulation modedetermination module 1434 may determine to switch the operational modeof robot arm 300 to the co-manipulation mode based on the identity ofthe surgical instrument.

Haptic mode determination module 1436 may be executed by processor 1402for analyzing the operating characteristics of robot arm 300 todetermine whether to switch the operational mode of robot arm 300 to thehaptic mode where the system applies an impedance to robot arm 300 viamotor interface module 1426 in an amount higher than applied in theco-manipulation mode, thereby making movement of robot arm 300responsive to movement at the handle of the surgical instrument moreviscous in the co-manipulation mode. For example, haptic modedetermination module 1436 may determine to switch the operational modeof robot arm 300 to the haptic mode if at least a portion of robot arm300 and/or the surgical instrument attached thereto is within apredefined virtual haptic boundary. Specifically, a virtual hapticboundary may be established by the system, such that the robot arm orthe surgical instrument coupled thereto should not breach the boundary.For example, a virtual boundary may be established at the surface of thepatient to prevent any portion of the robot arms or the instrumentssupported by the robot arms from contacting the patient, except throughthe one or more trocars. Similarly, the virtual haptic boundary mayinclude a haptic funnel to help guide the instrument into the patient asthe operator inserts the instrument into a trocar port. Accordingly,based on position data of robot arm 300 and/or the surgical instrumentcoupled thereto, e.g., received by robot arm position determinationmodule 1418 and/or trocar position detection module 1420, haptic modedetermination module 1436 may determine if robot arm 300 and/or thesurgical instrument is within the predefined virtual haptic boundary,and accordingly transition robot arm 300 to the haptic mode whereprocessor 1402 may instruct associated motors to apply an effectiveamount of impedance to the joints of robot arm 300 perceivable by theoperator to communicate to the operator the virtual haptic boundary.Accordingly, the viscosity of robot arm 300 observed by the operatorwill be much higher than in co-manipulation mode. In some embodiments,haptic mode determination module 1436 may determine to switch theoperational mode of robot arm 300 to the haptic mode based on theidentity of the surgical instrument.

Robotic assist mode determination module 1438 may be executed byprocessor 1402 for analyzing the operating characteristics of robot arm300 to determine whether to switch the operational mode of robot arm 300to the robotic assist mode where processor 1402 may instruct associatedmotors via motor interface module 1426 to cause movement ofcorresponding link and joints of robot arm 300 to achieve a desiredoutcome. For example, robotic assist mode determination module 1438 maydetermine to switch the operational mode of robot arm 300 to the roboticassist mode if a predefined condition exists based on data obtainedfrom, e.g., optical scanner interface module 1428.

For example, robotic assist mode determination module 1438 may determinethat a condition exists, e.g., the field of view of a laparoscopecoupled to robot arm 300 or optical scanner 1100 is not optimal for agiven surgical procedure, e.g., due to blocking by the surgeon orassistant or another component of the system, based on image dataobtained from the laparoscope or optical scanner 1100 via opticalscanner interface module 1428, such that the robot arm coupled to thelaparoscope or optical scanner 1100 should be repositioned or zoomin/out to optimize the field of view of the surgical site for theoperator. Thus, in robotic assist mode, processor 1402 may instructrobot arm 300, either automatically/quasi-automatically or responsive touser input by the operator, to move to reposition the laparoscope and/orcause the laparoscope to zoom in or zoom out, or to increase aresolution of an image, or otherwise. For example, the user input by theoperator may be determined by gesture detection module 1430, asdescribed above, such that movement of the robot arm or a surgicalinstrument in a predefined gestural pattern in a first direction causesthe endoscope to increase resolution or magnification and in a seconddirection causes the endoscope to decrease resolution or magnification,and movement in another predefined gestural pattern causes the robot armholding the laparoscope to retract away from the patient’s body.

In addition, robotic assist mode determination module 1438 may determinethat a condition exists, e.g., that one or more trocars are not in anoptimal position, for example, due to movement of the patient, such thatrobot arm 300 should be repositioned to maintain the trocar in theoptimal position, e.g., in an approximate center of the movement rangeof robot arm 300, thereby minimizing the risk of reaching a joint limitof the robot arm during a procedure. Thus, in robotic assist mode,processor 1402 may instruct system to reposition robot arm 300, e.g.,via vertical/horizontal adjustment by platform 100 or via the joints andlinks of robot arm 300, to better align the surgical instrumentworkspace.

Robotic assist mode determination module 1438 may determine that acondition exists, e.g., the distance between an object and robot arm 300is within a predetermined threshold, based on image data obtained fromthe laparoscope or optical scanner 1100 via optical scanner interfacemodule 1428, such that the robot arm should be frozen to avoid collisionwith the object. Thus, in robotic assist mode, processor 1402 mayinstruct robot arm 300 apply the brakes to slow down the robot arm orinhibit or prevent movement within a predetermined distance from theother object.

Robotic assist mode determination module 1438 further may determine thata condition exists, e.g., robot arm 300 is in an extended position for aperiod of time exceeding a predetermined threshold during a surgicalprocedure, such that the robot arm should be repositioned to provide theuser more available workspace in the vicinity of the surgical instrumentcoupled to the extended robot arm. Thus, in robotic assist mode,processor 1402 may instruct the system to reposition robot arm 300,e.g., via vertical/horizontal adjustment by platform 100 and/or via thejoints and links of robot arm 300, to move robot arm 300 closer to thesurgical instrument.

Fault detection module 1440 may be executed by processor 1402 foranalyzing the data indicative of the operating characteristics of thesystem, e.g. position data generated by robot arm position determinationmodule 1418 and/or trocar position detection module 1420 and/or forcemeasurement calculated by force detection module 1422, to detect whethera fault condition is present. For example, fault detection module 1440may a fault condition of the system and determine whether the faultcondition is a “minor fault,” a “major fault,” or a “critical fault,”wherein each category of fault condition may be cleared in a differentpredefined manner.

For example, fault detection module 1440 may detect a minor faultcondition such as robot arm 300 being moved with a velocity exceeding apredetermined velocity threshold, which may be cleared, e.g., by slowingdown the movement of robot arm 300. In some embodiments, the system mayautomatically apply additional impedance to robot arm 300 when robot arm300 is moving too fast to thereby force the operator to slow downmovement of robot arm 300. Moreover, fault detection module 1440 maydetect a major fault condition such as an inadvertent bump of robot arm300 as indicated by a large force applied to robot arm 300 by a personother than the operator. In response to detection of a major faultcondition, fault detection module 1440 may actuate the braking mechanismassociate with each motorized joint of robot arm 300 (or at least thejoints associated with the major fault condition), to thereby freezerobot arm 300 and inhibit further movement of robot arm 300. Such amajor fault condition may be cleared by the operator actuating a “clear”option displayed on user interface 1408. Fault detection module 1440 maydetect a critical fault condition such as redundant encoders associatedwith a given joint of robot arm 300 generating different angulationmeasurements with a delta exceeding a predetermined threshold. Inresponse to detection of a critical fault condition, fault detectionmodule 1440 may actuate the braking mechanism associate with eachmotorized joint of robot arm 300 to thereby freeze robot arm 300 andinhibit further movement of robot arm 300. Such a critical faultcondition may be cleared by the operator restarting the system. Uponrestart of the system, if the critical fault condition is still detectedby fault detection module 1440, robot arm 300 will remain frozen untilthe critical fault condition is cleared.

Indicator interface module 1442 may be executed by processor 1402 forcausing indicators 334 to communicate the state of the system, e.g., theoperational mode of robot arm 300, to the operator or other users, basedon, for example, determinations made by passive mode determinationmodule 1432, co-manipulation mode determination module 1434, haptic modedetermination module 1436, and/or robotic assist mode determinationmodule 1438. For example, indicator interface module 1442 may causeindicators 334 to illuminate in specific color light associated with aspecific state of the system. For example, indicator interface module1442 may cause indicators 334 to illuminate in a first color (e.g.,yellow) to indicate that no surgical instrument is attached to the robotarm, and that the robot arm may be moved freely such that the systemcompensates for the mass of the robot arm; in a second color (e.g.,purple) to indicate that a surgical tool is attached to the robot arm,and that the robot arm may be moved freely such that the systemcompensates for the mass of the robot arm and the mass of the surgicalinstrument coupled to the robot arm; in a third color (e.g., blue) toindicate that a surgical instrument is attached to the robot arm, andthat the robot arm is in the passive mode as determined by passive modedetermination module 1432; in a fourth color (e.g., pulsing orange) toindicate that at least a portion of the robot arm and/or the surgicalinstrument attached thereto is within the virtual haptic boundary, e.g.,1.4 m or more above the ground; in a fifth color (e.g., pulsing red) toindicate that a fault has been detected by the system by fault detectionmodule 1440. As will be understood by a person having ordinary skill inthe art, different colors and patterns may be communicated by indicators334 to indicate the states of the system described above.

Additionally, indicators 334 may be illuminated in other distinct colorsand/or patterns to communicate additional maneuvers by robot arm 300,e.g., when robot arm 300 retracts the surgical arm in the robotic assistmode, or performs another robotically-assisted maneuver in the roboticassist mode. As described above, indicators 334 further may includedevices for emitting other alerts such as an audible alert or textalert. Accordingly, indicator interface module 1442 may cause indicators334 to communicate the state of the system to the operator using audioor text, as well as or instead of light. For example, indicatorinterface module 1442 may cause one or more speakers to emit an audiblealert that changes in, e.g., amplitude and/or frequency, as robot arm300 approaches a potential collision with one or more objects/personswithin the operating room.

Additionally or alternatively, indicator interface module 1442 maycommunicate the state of the system, e.g., transition fromco-manipulation mode to passive mode, via haptic feedback at the distalend of robot arm 300, and accordingly on the surgical instrument coupledthereto. For example, when the surgical instrument is held in a positionfor the predetermined dwell time such that the system switches topassive mode, the user may feel a vibration at the surgical instrumentindicating that the system has transitioned to passive mode and that theuser may let go of the surgical instrument. As another example, the usermay feel a vibration after the surgical instrument is coupled to thecoupler body to indicate that the surgical instrument is successfullycoupled to the robot arm. The vibration may be strong enough to be feltby the user, but weak enough such that any movement at the distal tip ofthe surgical instrument resulting therefrom is negligible.

Fatigue detection module 1444 may be executed by processor 1402 fordetecting user fatigue that may occur during operation of robot arm 300in a surgical procedure, as described in further detail below withregard to FIG. 25 . For example, based on data from, e.g., robot armposition determination module 1418, force detection module 1422,impedance calculation module 1424, fatigue detection module 1444 maydetermine the level of fatigue of the operator using the surgicalinstrument coupled to robot arm 300, and compare the level of fatiguewith a predetermined fatigue threshold. For example, fatigue detectionmodule 1444 may assess an overall score for a given procedure todetermine the level of fatigue based on, e.g., operator hand tremor,distance/minimum path travelled by the instrument tip, time to achieveprocedure steps, and/or time to complete the procedure. Based on thedata generated by fatigue detection module 1444, impedance calculationmodule 1422 may determine an amount of impedance necessary to apply torobot arm 300 to, e.g., reduce tremor of the operator, such that motorinterface module 1426 may cause the associated motors to apply therequisite impedance to robot arm 300. Moreover, based on the datagenerated by fatigue detection module 1444, motor interface module 1426may cause the associated motors to move the links of robot arm 300 toguide the operator’s manipulation of the surgical instrument attachedthereto.

The co-manipulation surgical robot systems described herein may includeadditional modules within memory 1410 of platform 200 for executingadditional tasks based on the data obtained. For example, the system maydetermine that a surgical instrument has been attached to robot arm 300by detecting a rapid or sudden change in force (a “snapping motion”)applied to robot, e.g., due to the attraction force of the magneticconnection between the coupler body and coupler interface 400, via forcedetection module 1422. For example, the attractive forces of the magnetson the coupler body and coupler interface 400 may cause a suddenmovement on at least an end portion of the robot arm, and/or a suddenrotation of the last joint of the robot arm when the magnets arealigning. Accordingly, this sudden movement may be detected and maytrigger surgical instrument identification module 1412 to determine thatan instrument has been attached or detached from the robot arm.Similarly, surgical instrument identification module 1412 may determinethat the surgical instrument has been detached from robot arm 300, e.g.,when subsequent motions of the distal end of robot arm 300 areaccompanied by little to no rotation in the distal-most joint of robotarm 300.

Additionally, the system may determine if the surgical instrument hasbeen detached from robot arm 300 based on data indicative of theposition of the distal end of robot arm 300 relative to the trocar pointgenerated by trocar position detection module 1420, as well as thedirection of an instrument shaft and/or an orientation of thedistal-most link of robot arm 300, e.g., distal wrist link 316. Forexample, if the instrument is pointing directly at the trocar, thenthere is a higher probability that a tool is attached to the robot arm.Moreover, axis Q7 of robot arm 300 may indicate the pointing directionof the instrument and, if the instrument is passing through the trocarport, the distal wrist link 316 will point in a direction of the trocarport. Therefore, if distal wrist link 316 is not pointing toward thetrocar port, then the system may determine that the robot arm is notsupporting an instrument or the instrument is not advanced through thetrocar port. For example, when an instrument is detached from robot arm300 and robot arm 300 is moved, the computed direction of the instrumentshaft (e.g., the direction that the instrument would point if attachedto robot arm 300) may no longer point to the trocar entry point andlikely will not point to the trocar entry point. Accordingly, the mayalert a user if the system determines that no tool is coupled with robotarm 300, e.g., via indicators 334.

In addition, the system may identify when a user may be attempting toremove or decouple a surgical instrument from robot arm 300 and adjustthe removal force required to decouple the surgical instrument, andaccordingly the coupler body, from coupler interface 400. For example,where one or more magnets are used to provide a biasing force to biasthe surgical coupler body to the coupler interface, a force greater thanthe attraction force provided by the one or more magnets in a directionopposing the force provided by the one or more magnets must be exertedon the surgical instrument and/or the coupler body that is coupled withthe surgical instrument to overcome the attracting force and decouplethe coupler body and surgical instrument from the coupler interface. Forexample, the removal force may be 30-60 Newtons.

Moreover, the system may gather and analyze telemetry data regardingforces being applied to the robot arm to assess or estimate whether auser is attempting to remove a tool from the robot arm and, if so,reduce the coupling force between the coupler body and the couplerinterface to make it easier for the user to disengage the surgicalinstrument from the robot arm. For example, the coupling/removal forcemay be reduced by 50-80%. Based on historical data and user feedback, aswell as on data such as whether a user replaces the instrument withoutadjusting a location of the instrument, which could indicate inadvertentremoval of the instrument, the system may estimate the optimal times toreduce a coupling force between the coupler body and the couplerinterface. Moreover, the coupling force may be increased duringoperation to prevent inadvertent removal of surgical instrument from therobot arm.

Additionally, the system may determine an optimum positioning of robotarms 300 and its joints, the surgical instruments coupled with the robotarms, or other components of the robot arms and/or the system based ondata obtained from the optical scanning devices used with the system,and provide guidance to the operator of the system to achieve theoptimum positioning. Data indicative of the optimum positioning furthermay be used by processor 1402 to instruct the motors to causecorresponding links and joints of robot arm 300 to move, e.g., inrobotic assist mode, to automatically reposition robot arm 300 and/orthe optical scanning devices in the optimum position, e.g., during thesetup stage or thereafter.

In addition, the system may collect data from sensors, e.g., positiondata of robot arm 300 or the surgical instrument attached thereto viathe encoders or optical scanning devices and/or position data of theoperator via body sensors or optical scanning devices, during aprocedure, e.g., during setup or operation of robot arm 300, such thatprocessor 1402 may detect deviations of movements or processes of thecurrent user as compared to a model or optimal movement pattern, andcommunicate the deviations to the current user in real-time. Forexample, processor 1402 may cause a monitor to display the deviations tothe current user in real-time, as well as the optimal and/or actualmovement pattern. Additionally, or alternatively, indicator interfacemodule 1440 may cause indicators 334 to indicate deviations from themodel or optimal movement pattern, e.g., by illuminating a specificcolor and/or in a specific pattern. Additionally, or alternatively,motor interface module 1426 may apply impedance to robot arm 300perceivable by the operator as haptic feedback including vibrations,restrictions on movement, or sensations to indicate deviations from themodel or optimal movement pattern. Accordingly, the system may be usedas a training tool for new users as such data may be used to optimizethe position of a surgical device in real-time.

The system further may analyze the depth map generated by the opticalscanning devices and cluster different groups of (depth) pixels intounique objects, a process which is referred to as object segmentation.Examples of such algorithms for segmentation may include: matchingacquired depth map data to a known template of an object to segment;using a combination of depth and RGB color image to identify and isolaterelevant pixels for the object; and/or machine learning algorithmstrained on a real or synthetic dataset to objects to identify andsegment. Examples of such segmentation on a depth map may include:locating the robot arms or determining the position of the robot arms;identifying patient ports (e.g., trocar ports) and determining adistance from the instruments to the trocar ports; identifying thesurgeon and distinguishing the surgeon from other operators in the room;and/or identifying the surgeon in the sensor’s field of view. Moreover,the system may use object segmentation algorithms to uniquely identifythe surgeon and track the surgeon with respect to, for example, asurgical table, a patient, one or more robot arms, etc. In addition, thesystem may use object segmentation algorithms to determine if a surgeonis touching or handling either of the robot arms and, if so, identifywhich robot arm is being touched or handled by the surgeon. The systemfurther may use object segmentation to locate the surgical instrumentand the distal end of the robot arm in 3D space, such that the systemmay determine whether the surgical instrument is attached to the distalend of the robot arm, e.g., based on proximity between the surgicalinstrument and the distal end of the robot arm.

Referring now to FIG. 15 , operation 1500 of the co-manipulationsurgical robot systems described herein is provided. As shown in FIG. 15, at step 1502, the operator may couple a selected surgical instrumentto coupler interface 400 of robot arm 300 via a coupler body, e.g.,coupler body 500, 600, 700. As described above, the operator may selecta coupler body sized and shaped to couple with the selected surgicalinstrument, e.g., based on the elongated shaft diameter of the surgicalinstrument. When the surgical instrument and coupler body are ready tobe coupled to robot arm 300, the operator may load the calibration fileof the selected surgical instrument, e.g., via user interface 1408, suchthat information associated with the selected surgical instrument, e.g.,a laparoscope or retractor, is loaded into the system. For example, theoperator may select the calibration file from a database of calibrationfiles for a variety of surgical instruments. The calibration files maybe stored from previous procedures, and may be pre-loaded to includecalibration files of commonly used laparoscopic instruments.

If the calibration file for the selected surgical instrument is notavailable in the database, the operator may self-calibrate the surgicalinstrument using the system. For example, FIG. 16 illustrates surgicalinstrument calibration process 1600 for calibrating a surgicalinstrument, e.g., to determine the center of mass of the surgicalinstrument, which may be used in calculating accurate force measurementson the surgical instrument and robot arm 300 during operation. At step1601, the operator may actuate the “startup” option on user interface1408. At step, 1602, the operator may select the “load tool calibration”to begin the calibration process. At step 1603, the system does notapply any impedance to robot arm 300 for gravity compensation of asurgical instrument. The system may apply impedance to robot arm 300 toaccount for the weight of robot arm 300, e.g., to prevent robot arm 300from dropping to the ground. At step 1604, the surgical instrument iscoupled to coupler interface 400 of robot arm 300 via the appropriatesized coupler body, which may cause wrist portion 411 of robot arm 300to rotate about axis Q7 to engage with the coupler body.

At step 1605, the system compensates for the gravity of the surgicalinstrument and the force applied by the hand of the operator, e.g., bymeasuring the force applied to the distal end of robot arm 300 due tothe mass of the surgical instrument. As described above, the forceapplied to the distal end of robot arm 300 may be measured by measuringthe motor current across the motors disposed in the base of robot arm300. If the system overcompensates for the gravity of the surgicalinstrument, at step 1606, robot arm 300 may “runaway”, e.g., driftupward. The runaway effect may be detected at step 1607, and at step1608, indicators 334 may blink to indicate to the operator of therunaway. At step 1609, the system may identify the runaway as a minorfault, and accordingly apply additional impedance to robot arm 300 andfreeze robot arm 300 when robot arm 300 slows down before removing theadditional impedance. Once the minor fault is addressed, calibrationprocess 1600 may return to step 1603.

After step 1605, when the system compensates for the gravity of thesurgical instrument, if the surgical instrument is detached, eitheraccidentally or manually by the operator at step 1611, at step 1610, thesystem detected the detachment of the surgical instrument from robot arm300. As a result, the system will stop compensating for the gravity ofthe surgical instrument, and calibration process 1600 may return to step1603. After step 1605, when the system compensates for the gravity ofthe surgical instrument, calibration process 1600 is ready to entercalibration mode at step 1612. For example, the operator may initiatecalibration mode via user interface 1408 at step 1613. At step 1614, thesystem may indicate to the operator, e.g., via user interface 1408and/or blinking of indicators 334, that it is safe to let go of surgicalinstrument, such that the operator may let go of the surgical instrumentat step 1616. At step 1615, the system calibrations the surgicalinstrument.

Referring again to FIG. 15 , when the surgical instrument and couplerbody are ready to be coupled to robot arm 300, and the appropriatecalibration file is loaded, the operator may easily place the couplerbody near coupler interface 400, such that the magnetic connectionbetween the coupler body and coupler interface 400 automatically alignsand coupled the surgical instrument to robot arm 300. The system willnow accurately compensate for the gravity of the selected surgicalinstrument. At step 1504, the user may use the co-manipulation surgicalsystem by freely manipulating the surgical instrument coupled to robotarm 300 in the ordinary manner that the operator would without robot arm300 coupled thereto. As shown in FIG. 15 , as the operator manipulatesthe surgical instrument, and accordingly robot arm 300 coupled thereto,the system may automatically switch between, e.g., co-manipulation mode1506, passive mode 1508, haptic mode 1510, and robotic assist mode 1512(collectively referred to as “operational modes”), upon detection ofpredefined conditions, as described below with regard to FIG. 17 . Insome embodiments, the system may automatically switch between onlyco-manipulation mode 1506, passive mode 1508, and haptic mode 1510. Insome embodiments, the operator may select which operational mode to setthe system in prior to using the co-manipulation surgical system at step1504.

For example, an operator may exert a particular force on the distal endof robot arm 300, e.g. by manipulating the surgical instrument coupledto robot arm 300, to indicate that the operator wishes to change theoperational mode of the particular robot arm. Sensors and/or motorcurrent readings may be used to detect the force applied to the distalend of robot arm 300 and to determine if the force matches a predefinedforce signature associated with an operational change, e.g., bycomparing the force with one or more predefined force signatures storedin the system. If there is a match, then the system may change theoperational mode of the robot arm to the particular operational modethat matches the force signature.

As described above, during operation of the co-manipulation surgicalsystem, the system may continuously monitor the robot arm and forcesapplied thereto to detect predefined conditions that require switchingthe operational modes of the system, as described in method 1700 of FIG.17 . As shown in FIG. 17 , at step 1702, the system continuouslycollects data related to a first operating characteristic of the robotarm and/or of the surgical instrument coupled with the robot arm. Forexample, as described above, the system may measure motor current of themotors operatively coupled to the joints of the robot arm as well asangulations of the links of the robot arm based on measurements by theencoders of the robot arm to calculate the positon of the robot arm andthe surgical instrument as well as the forces acting on any portion ofthe robot arm as well as on the surgical instrument, if any, in realtime. At step 1704, the system may analyze the data related to the firstoperating characteristic to determine if a first condition is present.For example, based on the position and force data of the robot armand/or surgical instrument, the system may determine if the movement ofthe robot arm due to movement of the surgical instrument coupled theretois within a predetermined movement threshold of the robot arm for aperiod of time longer than the predetermined dwell time of the robotarm. Upon detection of this first condition, at step 1706, the systemmay modify a first operating parameter of the robot arm. For example,the system may switch the operational mode of the robot arm to thepassive mode, where the robot arm maintains the surgical instrument in astatic position.

For example, a first robot arm may be coupled to a laparoscope, and theoperator may manipulate the laparoscope within the patient until adesirable field of view is provided by the laparoscope, e.g., via amonitor displaying the image feed from the laparoscope. In order tofreely move the laparoscope coupled to the first robot arm in theco-manipulation mode, the operator must apply a sufficient force to thelaparoscope that exceeds a predetermined force threshold. Thepredetermined force threshold should be low enough such that it does notrequire much force by the operator to freely move the laparoscope.Moreover, the predetermined force threshold may be selected so as toresist inadvertent movement away from the passive mode. As the operatorfreely moves the laparoscope in the co-manipulation mode, as describedabove, the system will apply enough impedance to the first robot arm tocompensate for the effects of mass (i.e., inertia) and/or gravity of thefirst robot arm and the laparoscope during the movement, such that amass or weight of the first robot arm is not detectable by the operatoror is otherwise significantly attenuated. In some embodiments, if whenthe operator couples the laparoscope to the first robot arm, thelaparoscope is not already positioned within the body of the patient,the system may determine that there are no external forces acting on thesurgical instrument and may automatically switch the first robot arm tothe haptic mode in order to guide the operator to move the laparoscopeto the appropriate location through the trocar port, e.g., via a virtualhaptic funnel established about the trocar port.

When the laparoscope is in the desired position relative to the patientand the surgical site within the patient, the system will automaticallyswitch from co-manipulation mode to passive mode upon detection thatmovement of the first robot arm due to movement of the surgicalinstrument is within a predetermined movement threshold for a period oftime exceeding a predetermined dwell time. For example, upon reachingthe desired position, the operator will hold the laparoscope in thedesired position, e.g., for at least a quarter of the second. Thus, ifthe predetermined dwell time is a quarter of a second, holding thelaparoscope in the desired position for any longer than thepredetermined dwell period will cause the system to automatically switchto passive mode. Moreover, as the operator may not be able to hold thelaparoscope perfectly still, at least some movement of the laparoscopeis permitted for the duration of the predetermined dwell time to enterinto the passive mode. As described above, in passive mode, the firstrobot arm will hold the laparoscope in a static position, e.g., by thesystem applying enough impedance to the first robot arm to compensatefor all external forces acting on the laparoscope.

Similarly, a second robot arm may be coupled to a retractor, and theoperator may freely manipulate the retractor within the patient in theco-manipulation mode, e.g., to grasp tissue within the patient andretract the tissue to provide a clear field of view of the surgical siteby the laparoscope coupled to the first robot arm, by applying asufficient force to the second robot arm due to force applied at theretractor exceeding the predetermined force threshold of the secondrobot arm. As the operator grasps/lifts/retracts the tissue withretractor, the system may only compensate for the gravity of the secondrobot arm and/or the instrument and not of the tissue being grasped,such that the operator may feel any other forces acting on theretractor, including without limitation the forces acting on theinstrument from the tissue. In this optional configuration. Accordingly,the haptics associated with the tissue being grasped may be preserved.

When the retractor sufficiently grasps and retracts the tissue, thesystem may automatically transition to the passive mode upon theoperator holding the retractor in position, e.g., with movement notexceeding a predetermined movement threshold of the second robot arm,for a period of time exceeding the predetermined dwell period of thesecond robot arm. Accordingly, when the retractor is retracting thetissue within the patient in the passive mode, the second robot arm willaccount for the mass of the tissue in addition to the mass of theretractor and the second robot arm. Thus, the predetermined forcethreshold to cause the second robot arm to switch out of the passivemode must be greater than the force applied to second robot arm due toforce applied to the tip of the retractor by the tissue, such that ifthe force applied by the tissue to the surgical instrument exceeds thepredetermined first threshold of the second robot arm, the system willautomatically cause the second robot arm to switch out of the passivemode and into, e.g., the co-manipulation mode. However, thepredetermined force threshold should not be so high that it is verydifficult for the operator to move the retractor. As described above,the operator may adjust the predetermined force threshold via, e.g.,user interface 1408.

Upon retraction of the tissue via the retractor coupled to the secondrobot arm, the operator may need to readjust the field of view of thelaparoscope coupled to the first robot arm. Accordingly, the operatormay apply a force to the laparoscope that exceeds the predeterminedforce threshold of the first robot arm, such that the systemautomatically switches the first robot arm from the passive mode to theco-manipulation mode. When the new desired position of the laparoscopeis achieved, the first robot arm may automatically switch back to thepassive mode if the predefined conditions described above are met.Alternatively, to readjust the laparoscope or to reposition the links ofthe first robot arm to avoid potential collisions during thelaparoscopic procedure or to switch the laparoscope to a different robotarm altogether, the operator may elect to decouple the laparoscope,readjust the robot arm and/or laparoscope, and reattach the laparoscopeto the first robot arm (or to the other robot arm). Upon reattachment ofthe laparoscope to the first robot arm, the first robot arm mayautomatically switch to the passive mode if the predefined conditionsdescribed above are met.

Moreover, as the operator freely moves the retractor in theco-manipulation mode, e.g., prior to inserting the tip of the retractorthrough the trocar within the patient, if the operator moves the tip ofthe retractor too close to the patient’s skin away from the trocar port,and a virtual haptic boundary has been established by the system on theskin of the patient outside the trocar ports, the system mayautomatically switch to the haptic mode. Accordingly, the system mayapply an impedance to the second robot arm that is much higher than theimpedance applied to the second robot arm in co-manipulation mode toindicate to the operator that they are approaching or within the virtualhaptic boundary. For example, movement of the retractor by the operatormay feel much more viscous in the haptic mode. The system may remain inthe haptic mode until the operator moves the retractor out of thevirtual haptic boundary. In some embodiments, in the haptic mode, thesecond robot arm may reduce the effects of gravity, eliminate tremor ofthe instrument tip, and apply force feedback to avoid criticalstructures as defined by the virtual haptic boundary. Accordingly, thesystem does not replace the operator, but rather augments the operator’scapabilities through features such as gravity compensation, tremorremoval, haptic barriers, force feedback, etc.

In some embodiments, the system may switch the second robot arm to therobotic assist mode. For example, as the operator attempts to retractthe tissue, if more force is required to retract the tissue than theoperator is able or willing to apply to the retractor, the operator mayprovide user input to the system indicating that the operator wants thesecond robot arm to assist in the retraction of the tissue. For example,as described above, the operator may perform a predefined gesturalpattern that may be detected by, e.g., optical scanner 1100, such thatthe system switches the second robot arm to the robotic assist mode andcauses the motors of the second robot arm to move the second robot arm,and accordingly the retractor, to provide the additional force requiredto retract the tissue.

In addition, instead of manually manipulating the laparoscope coupled tothe first robot arm as described, the operator may provide another userinput to the system indicating that the operator wants the system toreposition the laparoscope. For example, if the operator is activelymanipulating a surgical scissor, which may or may not be coupled to arobot arm of the system, such that the tip of the surgical scissor iswithin the field of view of the laparoscope coupled to the first robotarm, the operator may perform a predefined gestural pattern with the tipof the surgical scissor, e.g., moving the surgical scissor quickly backin forth in a particular direction. The predefined gestural pattern ofthe surgical scissor may be captured as image data by the laparoscope,and based on the data, the system may detect and associated thepredefined gestural pattern with a predefined user input requiring thatthe system switch the first robot arm from the passive mode to therobotic assist mode, and cause the first robot arm to reposition itself,and accordingly the laparoscope, to adjust the field of view in thedirection of the pattern motion of the surgical scissor. As describedabove, additional gestural patterns may be performed via the surgicalscissor within the field of view of the laparoscope to cause the firstrobot arm to retract the laparoscope and/or to cause the laparoscopeitself to zoom in or zoom out or improve resolution. In someembodiments, based on the image data captured by the laparoscope, usingobject tracking of the additional tools in the field of view of thelaparoscope, e.g., the surgical scissors actively operated by theoperator, the system may cause the first robot arm coupled to thelaparoscope to automatically switch to the robotic assist mode and causethe first robot arm to reposition itself to adjust the field of view toensure that the tip of the surgical scissors remain within an optimumposition within the field of view of the laparoscope during theprocedure.

The operational mode of any one of the robot arms may be changedindependent of the operational mode of the other robot arms of thesystem. In addition, the operational parameters of each robot arm may betailored to the specific surgical instrument coupled thereto. Forexample, the predetermined force threshold for the robot arm coupled tothe retractor device may be higher than the predetermined forcethreshold for the robot arm coupled to the laparoscope, as the retractorwill endure higher forces during the procedure. The sensors, motors,etc. of the system may be active in all modes, but may act verydifferently in each mode, e.g., including acting as if inactive. As willbe understood by a person having ordinary skill in the art, the systemmay include more than two robot arms, such that the operator may couplea third surgical instrument, e.g., a grasper device, to a third robotarm and a fourth surgical instrument, e.g., a surgical scissor device,to a fourth robot arm for operation during the laparoscopic procedure.

In some embodiments, the operational mode of a robot arm may be changedresponsive to user input provided by the operated. For example, theoperator may selectively change the operational mode of the robot arm byactuating a button, dial, or switch located on the robot arm, a footpedal or foot switch, voice command, an input on a touchscreen, or usinggestures or force signatures as described above. In some embodiments,the operational mode of a robot arm may be changed based only on thecoupling of the surgical instrument to the coupler interface via thecoupler body. As described above, the system may automatically identifythe surgical instrument based on the coupling of the coupler body to thecoupler interface. Accordingly, based on the identity of the surgicalinstrument coupled to the robot arm, the system may automatically switchthe operational mode of the robot arm to a predetermined operationalmode, e.g., passive mode if the surgical instrument is an endoscope, orif the robot arm is already in the passive mode, the system will remainin the passive mode upon coupling of the endoscope with the robot arm.

Similarly, based on the identity of the surgical instrument uponattachment of the surgical instrument to the robot arm, the system mayautomatically switch the operational mode of the robot arm to theco-manipulation mode, e.g., is the surgical instrument identityindicates that it is a tool that will be actively operated by theoperator during the laparoscopic procedure. Additionally, based on theidentity of the surgical instrument upon attachment of the surgicalinstrument to the robot arm, the system may automatically switch theoperational mode of the robot arm to the robotic assist mode, e.g., ifthe surgical instrument identity indicates that it is a tool that theoperate desires to be completely robotically controlled such as anirrigation device. Accordingly, upon attachment of the irrigation deviceto the robot arm, the system will switch to the robotic assist mode andcause the robot arm to position the irrigation device in the desiredposition within the body.

Moreover, the system may be instructed by the operator, e.g., via userinterface 1408, to operate the robot arm in less than the fouroperational modes discussed above. For example, the operator maydeactivate any one of the operational modes for a give procedure. Insome embodiments, the system may cause the robot arm to operate in anadditional operational mode, such as a locking mode, which may besimilar to the passive mode, except that the predetermined forcethreshold of the robot arm to switch out of passive/locking mode may beso high that the robot arm is effectively frozen so as to protect therobot arm from inadvertently switching out of the passive/locking mode,e.g., to avoid movement due to inadvertent bumps of the robot arm. Inthis locking mode, if the force from the inadvertent bump issufficiently high to cause even a slight movement of the robot arm, thesystem may cause the robot arm to reposition itself to the position itwas in prior to the inadvertent bump.

In addition, when no surgical instrument is coupled to the distal end ofa robot arm of the system, the system is still capable of automaticallyswitching the operational modes of the robot arm responsive to movementof the robot arm by an operator upon detection of the predefinedconditions described above. Accordingly, the system will apply animpedance to the joints of the robot arm to compensate for the mass ofthe robot arm such that the robot arm may remain in a static positionwhen in the passive mode, and will permit the robot arm to be freelymoveably by the operator in the co-manipulation mode if the systemdetects that the force applied to the robot arm by the operator exceedsthe predetermined force threshold of the robot arm. Additionally, thesystem will switch the robot arm to the haptic mode if the operatorattempts to move any portion of the robot arm within a predefinedvirtual haptic barrier. At step 1514, when the laparoscopic procedure iscomplete, the operator may remove the surgical instruments from therespective robot arms.

Referring now to FIGS. 18A to 18C, force measurements during operationof robot arm 300 are provided. As described above, upon attachment ofthe surgical instrument to coupler interface 400 via the coupler bodycoupled to the surgical instrument, the orientation of the surgicalinstrument may be automatically determined based on the magneticconnection between the coupler interface and the coupler body. Moreover,as described above, the calibration file of the surgical instrumentcoupled to robot arm 300 loaded on the system may include information ofthe surgical instrument including, e.g., the mass of the surgicalinstrument, the center of mass of the surgical instrument, and thelength of the surgical instrument, such that distance D3 between thecenter of mass and the instrument tip may be derived. In addition, asdescribed above, the position of the surgical instrument at the trocar,e.g., where the surgical instrument enters the patient’s body, may becalculated in real-time, such that distance D2 between the center ofmass of the surgical instrument and the trocar may be derived in realtime. Additionally, as described above, the coupler body is preferablycoupled to the surgical instrument at a fixed, known position along theelongated shaft of the surgical instrument (which may be included in thecalibration file), e.g., adjacent to the proximal portion of thesurgical instrument, and thus distance D1 between the center of mass ofthe surgical instrument and the coupler body, e.g., the point ofattachment to the distal end of robot arm 300, may be derived.Alternatively or additionally, as described above, optical scanningdevices may be used determine any one of D1, D2, or D3.

As shown in FIG. 18A, when the surgical instrument is positioned throughtrocar Tr, without any additional external forces acting on the surgicalinstrument other than at trocar Tr, e.g., the surgical instrument is notlifting or retracting tissue within the patient, the force applied tothe surgical instrument at trocar Tr by the body wall (e.g., the “bodywall force” or the “trocar force”) may be calculated with the followingequation:

F_(eff) + W + F_(tr) = 0 =  > F_(tr) = −W − F_(eff)

Where F_(eff) is the force at the distal end of robot arm 300 (e.g., the“end-effector force” of robot arm 300), W is the weight vector of thesurgical instrument (=-mgz), and F_(tr) is the trocar force.Accordingly, F_(eff) is the desired force sent to the system, which isthe sum of all the forces generated in the algorithm pipeline including,e.g., gravity compensation, hold, etc.

As shown in FIG. 18B, when the surgical instrument is positioned throughtrocar Tr and holding/retracting tissue, such that an external force isapplied to the tip of the surgical instrument, there are two forces toresolve: F_(tr) and F_(tt). Accordingly, two equations are needed tosolve for the two unknown vectors, which may be the balances of forcesand also the balance of moments around the center of mass of thesurgical instrument, e.g., L_(cg).

W + F_(eff) + F_(tr) + F_(tt) = 0

F_(eff) × D1 + F_(tr) × D2 + F_(tt) × D3 = 0

Here, distances D1 and D3 are known as described above, and D2 may bederived based on the known position of the distal end of robot arm 300and the calculated position of trocar Tr. As shown in FIG. 18B, thecenter of mass L_(cg) of the surgical instrument is behind the point ofattachment of the coupler body to the distal end of robot arm 300.

As described above, the system may alert the operator if the forces,e.g., force F_(tt) applied to the tip of the instrument and/or forceF_(tr) applied by the instrument at the trocar using, are greater thanthe respective threshold forces, and accordingly freeze the system ifthe calculated force is greater than the threshold force, and/or reducethe force exerted at the trocar point at the body wall or at the tip ofthe instrument by automatically applying brakes or stopping forces torobot arm 300, by slowing or impeding further movement of the instrumentin the direction that would increase forces applied at the tip of theinstrument or the trocar, and/or automatically moving the robotic arm ina direction that reduces the force being exerted at the instrument tipand/or at the trocar point at the body wall.

Referring now to FIG. 20 , a high level example 2000 of the differentcombinations of data inputs for the various sensors and devices of thesystems disclosed herein, e.g., system 200, and the multiple featuresand capabilities that any implementations of the systems disclosedherein may have and can produce based at least in part on the multiplepossible data inputs is provided. As shown in FIG. 20 , someimplementations of the system may be configured to gather data from atleast three monitoring sources 2002, including telemetry from the system(which may include force data from the robot arms, position data fromthe robot arms, etc.), video from the laparoscopic tower, and/or datafrom optical scanner 1100. The data gathered from the monitoring sources2002 may undergo data processing steps 2004 using one or more processorsin the system. The data processing steps may include, e.g., data fusion(e.g., fusion of the data gathered from the monitoring sources 2002) anddata analysis, which may include algorithm computations. In addition,the data from the monitoring sources 2002 may undergo processing 2004for the development of system usability features 2006, system safetyfeatures 2008, and system performance features 2010. The system mayprovide the features in real-time. For example, the system usabilityfeatures may include identifying the surgeon and adjusting the platformheight based on the surgeon’s profile, detecting the skin surface of thepatient and creating a virtual boundary around the skin surface toprevent inadvertent contact with the skin surface of the patient,detecting an instrument type and automatically loading the calibrationfile appropriate for the particular instrument, etc. In addition, thesystem safety features may include displaying a virtual map of the areasurrounding platform 100, e.g., as an operator moves platform 100throughout the operating room, to provide the operator with a view ofthe area surrounding platform 100, such that the operator may avoidcollisions between platform 100 and any objects and/or persons withinthe area surrounding platform 100.

Referring to FIG. 21 , a schematic overview of the electrical componentsof the electrical system and connectivity 2100 of the system isprovided. This includes the flow of energy throughout the illustratedportion of the system, the ports that may be used for connectivity, andother details related to the various electronic components. For examplethe system may include non-real time computer 2102 that may be used toacquire data from the optical scanning devices and perform otherfunctions. Non-real time computer 2102 also may control the graphicaluser interface of the system for the surgeon to interact with. Asdescribed above, the graphical user interface may include a touchscreen. Non-real time computer 2102 may include, e.g., a 10th Gen Intel®Core™ i7-10700 processor, 32 GB of RAM (which can optionally be 2×16 GB,DDR4, 2933 Mhz), a standard keyboard and a 512 GB PCIe M.2 SSD +1TB SATA7200 RPM hard drive, a wireless and Bluetooth card such as the Killer™Wi-Fi 6 AX1650i (2×2) 802.11ax Wireless and Bluetooth 5.1, and/or aNVIDIA® GeForce RTX™ 2060 6GB GDDR6 graphics card. The system furthermay include real-time computer 2104 that may be used to operate andcontrol the robot arms and the related robot controllers and/or otherfunctions, such as acquiring data and information from the opticalscanning devices. Real-time computer 2104 may include, e.g., an IntelCore i7 (8th Gen) processor, 32 GB of RAM for memory, a 500 GB SDD harddrive, and/or two or more RJ45 connectors for Ethernet connectivity.

Referring now to FIG. 22 , a flow chart of process 2200 for theacquisition and processing of data from an optical scanning device isprovided. As shown in FIG. 22 , at step 2202, depth data may be acquiredfrom one or more optical scanning devices, e.g., optical scanner 1100.At step 2204, filtering/other signal processing algorithms may beperformed, e.g., median filter, Gaussian noise removal, anti-aliasingalgorithms, morphological operations, ambient light adjustments, etc. Atstep 2206, 3D object segmentation may be performed using, e.g., templatematching, machine learning, Brute force matching, color plus depthsegmentation, 2D-3D registration, pixel value thresholding, etc. At step2208, object coordinates may be transformed to task space. For example,transforming object coordinates to task space may include converting aposition and an orientation of an object from the optical scanningdevice’s coordinate frame to the coordinate frame of the task needed(e.g., a robot frame for robot control, a cart frame for system setup,etc.). Additionally or alternatively, transforming object coordinates totask space may include using known optical scanning device to thesupport platform (e.g., a cart) transformations, the surgical robottransformations, and/or the user interface screen transformations, andgenerating new transformations for specific tasks such as tracking thesurgeon’s body (e.g., face, hands, etc.) with respect to differentelements of the system (e.g., support platform, robot arms, screen,etc.), tracking the surgical table with respect to the cart platform,tracking patient orientation for system setup, tracking trocar portlocation and orientation for setup, and tracking the position ofoperating room staff for safety. At step 2210, the desired task may beperformed, e.g., moving the robot arms into the vicinity of thepatient/trocar port for easy setup, tracking operating room staff toensure the system only responds to surgeon commands, recording thesurgeon’s hand movements during different phases of surgery, etc.

In addition, FIG. 22 illustrates a flow chart of process 2212 for theacquisition and processing of data from an optical scanning device. Atstep 2214, depth data may be acquired from one or more optical scanningdevices, e.g., optical scanner 1100. At step 2216, specular noisefiltering may be performed. At step 2218, patient/trocar portsegmentation and identification may be performed. At step 2218, trackedport coordinates may be transformed to robot coordinate space. At step2222, the robot arms may be moved to a desired vicinity of thepatient/trocar port.

Referring now to FIG. 23 , an example data flow 2300 of the system isprovided. As shown in FIG. 23 , non-real-time computer 2302 may gatherdata from an optical scanning device, e.g., optical scanner 1100 and/orfrom a camera feed from a laparoscope. Non-real-time computer 2302 alsomay receive data from real-time computer 2308 having a robot controller,including telemetry information such as positions of the robot arms,forces applied to the various motors/sensors of the robot arms,operational mode information, etc. Non-real-time computer 2302 also mayreceive data from patient database 2310 having information specific tothe patient in the procedure including, e.g., CT scan data, relevanthealth conditions, and other information that may be desired by thesurgeon.

Non-real-time computer 2302 further may provide user feedback 2312 tothe user via user interface 2314. User feedback may include, e.g.,collision notifications, positioning information and/or recommendationsregarding the various components of the system, the operational modethat has been detected by the system, etc. Non-real-time computer 2302further may provide commands 2318, e.g., high level commands, toreal-time computer 2308. High-level commands may include, e.g., modechanges, trajectories, haptic barriers, user configurations, etc.Real-time computer 2308 may include robot controller 2320 programmed toprovide robot commands 2322, e.g., motion or force commands, to the oneor more robot arms 2324, e.g., robot arms 300. Robot controller 2320 mayreceive robot feedback data 2326, e.g., motion, force, and/or touchpointdata, etc., from the one or more robotic arms 2324.

Referring now to FIG. 25 , method 2500 for estimating user fatigueduring a surgical procedure using robot arm 300 is provided. Asdescribed above, the algorithms for gravity compensation, viscosity,and/or effects of mass may be used to account for user fatigue.Specifically, during a laparoscopic procedure, a surgeon may be subjectto fatigue and may experience hand tremor or erroneous tool motion forsurgical tools such as, e.g., scissors, needle drivers, cautery tools,graspers, as the procedure progresses. As shown in FIG. 25 , at step2502, the system may receive and monitor data indicative of theoperator’s performance, e.g. from optical scanner 1100 such as a LiDARcamera, robot telemetry, and/or an endoscope, during the surgicalprocedure while the operator maneuvers the surgical instruments coupledto robot arm 300. Learning from a large dataset of clinical proceduresand/or gathering and analyzing data during a procedure or a portion of aprocedure may allow the system to infer a level of competency of thesurgeon as the procedure progresses, at step 2504, and further may allowthe system to adapt algorithm parameters in order to help the surgeon tomove more effectively while co-manipulating the surgical instrumentsattached to the robot arm. For example, at step 2506, the system mayadjust one or more operating parameters of robot arm 300 to change itsbehavior. If the fatigue level goes above a specific threshold, at step2608, the system may warn the surgeon. In addition, ranking proceduresmay be used to allow the system to provide the surgeon a summary oftheir performance for a given procedure and show their overall progress,procedure after procedure.

In some embodiments, the system may collect data during a procedureindicative of at least one of operator hand tremor, distance/minimumpath travelled by the instrument tip, time to achieve procedure steps,and/or time to complete the procedure, and compare such data withthreshold or predefined values for each of the factors to determinewhether a magnitude of any one of the factors has reached a levelsufficient to cause the system to warn the operator and/or sufficient tocause the system to adjust one or more operating parameters to mitigatethe user’s fatigue. For example, the system may eliminate or reducetremor of the instrument tip by exerting forces on the instrument toincrease the impedance or viscosity of the instrument, to avoid criticalstructures, and/or to apply force feedback. User fatigue may beidentified when, for example, a procedure time increases beyond athreshold value for a particular procedure, the number of movements ofthe surgical instrument increases beyond a threshold value for aparticular procedure or otherwise indicates errant or uncontrolledmovements, if an operator moves an instrument into a haptic barrier apredefined number of times, if an operator exerts an excessive force onthe trocar one or a predetermined number of times, etc. As describedabove, such data may be collected using the sensors on the robot armsand/or one or more optical scanning devices. When a particular level ofuser fatigue is identified by the system, the system may increase aviscosity or impedance of the instrument and/or the robot arm associatedwith the instrument to reduce a magnitude of movements and/or a numberof movements of the surgical instrument and/or the robot arm.

Additionally, the system may collect data regarding the speed andfrequency with which the operator moves the variousinstruments/laparoscopes along with estimates of how much tremor isinvolved in the movements, estimate the required added viscosity toreduce tremors while not hindering their motions or adding unnecessaryfatigue to the operator. In some embodiments, a controller of robot arm300 may iteratively adjust a viscosity value for a particularinstrument, collect data related to the movement of the instrument, andto assess whether an additional adjustment is needed to the viscosityapplied to the instrument. Moreover, the system may use additionalalgorithms to adopt an iterative approach to optimizing a particularoperational characteristic or parameter of robot arm 300, includingcollecting data related to a particular operational characteristic orparameter, changing operational characteristic or parameter, collectingadditional data related to the operational characteristic or parameter,and analyzing the data to determine if additional changes to theoperational characteristic or parameter should be made, which may bebased on, e.g., deviations between the actual data values and preferredor optimal values of an operational characteristic or parameter.

Referring now to FIG. 26 , dataflow 2600 of a distributed network ofco-manipulation surgical robot systems is provided. For example, adistributed network of co-manipulation robotic (“cobot”) surgicalsystems may be used in multiple hospitals, each of which may beconnected to an online database. This arrangement may provideconsiderably more data and user information that may be used by any ofthe cobot systems in operation. The systems may aggregate the data fromthe distributed network of systems to identify the optimum configurationbased on factors such as procedure type, surgeon experience, patientattributes etc. Through analytics or clinician input, the cobot systemsmay identify a routine procedure versus a procedure that may be morecomplicated. This information may be used to provide advice or guidanceto novice surgeons.

Moreover, centralizing procedure data may enable the running of largedata analytics on a wide range of clinical procedures coming fromdifferent users. Analysis of data may result in optimized settings for aspecific procedure, including, e.g., optimized system positioning,optimal ports placement, optimal algorithms settings for each robot armand/or detection of procedure abnormalities (e.g., excessive force,time, bleeding, etc.). These optimal settings or parameters may dependon patient and tool characteristics. As described above, a surgeon mayload and use optimal settings from another surgeon or group of surgeons.This way, an optimal setup may be achieved depending on, e.g., thesurgeon’s level of expertise. To keep track of the various users in thedistributed network of cobot systems, it may be beneficial to identifyeach user. As such, the user may log into the cobot system and accesstheir profile online as necessary. This way the user may have access totheir profile anywhere and will be able to perform a clinical procedurewith their settings at a different hospital location.

An example user profile may contain the user’s specific settings andinformation, including, e.g., username; level of expertise; differentprocedures performed, and/or region of clinical practice. In addition,the clinical procedure may require a user to store specific settingssuch as clinical procedure (e.g., cholecystectomy, hernia, etc.), tableorientation and height, preferred port placement, settings per assistantarm for each algorithm, patient characteristics (e.g., BMI, age, sex),and/or surgical tools characteristics and specifications (e.g., weights,length, center of gravity, etc.). The user may be able to enable his ownprofile, and optionally may enable another user’s profile, such as theprofile of a peer, the most representative profile of a surgeon of theuser’s area of practice, the most representative profile of a surgeonwith a specific level of expertise, and/or the recommended profileaccording to patient characteristics.

The identification of a user may be performed via password, RFID key,facial recognition, etc. Learning from a large number of procedures mayresult in a greater level of optimization of the cobot system setup fora given procedure. This may include, e.g., cart position, individualrobot arm position, surgical table height and orientation, portplacement, and/or setup joints position. These settings may be based onpatient height, weight, and sex, and further may be interdependent. Forexample, the optimal port placement may depend on patient tableorientation.

Additionally, a clinical procedure may be described as a sequence ofclinical procedures steps. Learning these different steps may allow thecobot system to infer in real time the actual step for a givenprocedure. For example learning clinical steps from procedures may allowor enable: adjustment of algorithm settings, the system to give thepractical custom reminders, the system to notify staff of an estimateprocedure end time, the system to alert staff if necessary equipment isnot available in the room, and/or the system to alert staff of theoccurrence of an emergency situation.

During a clinical procedure, the surgeon will often realize simple androutine surgical tasks such as grasping, retracting, cutting etc.Learning these different tasks may allow the cobot system to infer inreal time preferences and habits of the surgeon regarding a sequence ofa procedure in real time. Some algorithms of the cobot system may betuned (i.e., adjusted and optimized) during the procedure based on thissequence recognition and help the user to be better at this simplesurgical task. An example of such a task is the automated retraction ofa liver during a gall bladder procedure. By aggregating the informationover many cases, the optimized force vectors may be developed.

Further, some complications may occur during a clinical procedure thatmay result in unexpected steps or surgical acts. Learning how todiscriminate these unexpected events would help the cobot system toenable some specific safety features. In case of emergency, the robotarms may be stopped or motion restricted depending on the level ofemergency detected by the system.

Referring now to FIGS. 27A to 27D, setup of the co-manipulation surgicalsystem is provided. Platform 2700 may be constructed similar to platform100, such that platform 2700 supports one or more robot arms, e.g.,robot arm 300 a′ and robot arm 300 b′, and may cause the robot arms tomove relative to platform 2700. As shown in FIG. 27A, platform 2700 maybe moved to a desirable position relative to patient table PT by a user,e.g., via wheels 104′, while robot arms 300 a′, 300 b′ are in theirrespective stowed configurations.

As platform 2700 is being moved toward the patient, the scene may bedirectly observed by a depth mapping sensor, e.g., optical scanner1100′, which may be mounted on platform 2700. From the depth mapsobserved and generated by optical scanner 1100′, key features may beidentified such as, for example, the height and/or location of patienttable PT, the surface of the patient’s abdomen, position and othercharacteristics of the surgeon, including the surgeon’s height, and thetrocar port(s), the base of robot arms 300 a′, 300 b′, e.g., baseportions 302 a′, 302 b′ and shoulder portions 304 a′, 304 b′, robot arms300 a′, 300 b′, and/or one or more surgical instruments coupled with therobot arms. Identification of such key features may be carried out usingstandard computer vision techniques such as template matching, featuretracking, edge detection, etc. As each feature is registered, itsposition and orientation may be assigned a local co-ordinate system andtransformed into the global co-ordinate system the system using standardtransformation matrices. Once all features are transformed into a singleglobal co-ordinate system, an optimization algorithm, e.g., leastsquares and gradient descent, may be used to identify the mostappropriate vertical and horizontal positions of robot arms 300 a′, 300b′, which may be adjusted via platform 2700, to maximize the workspaceof the robot arms with respect to the insertion point on the patient.The optimal workspace may be dependent on the surgical operation to beperformed and/or the surgeon’s preferred position.

As shown in FIG. 27B, when platform 2700 is in its desired positionrelative to patient table PT, such that wheels 104′ are locked, robotarms 300 a′, 300 b′ may be extended away from their respective stowedconfigurations. As shown in FIG. 27C, the vertical position of the robotarms relative to platform 2700 may be adjusted to the desired position,and as shown in FIG. 27D, the horizontal position of the robot armsrelative to platform 2700 may be adjusted to the desired position.

Referring now to FIGS. 28A to 28D, screenshots of exemplary graphicaluser interface 2800 are provided. Exemplary graphical user interface2800 may be configurable by a user and may be integrated with display110. FIG. 28A illustrates an exemplary start menu. The operator mayinitiate operation of the co-manipulation system by actuating the“start” option. FIG. 28B illustrates an exemplary system setup screen.As shown in FIG. 28B, when the system includes two robot arms, graphicaluser interface 2800 may identify which robot arm is to be used withwhich instrument, e.g., retractor arm 2806 and endoscope arm 2808, aswell as the procedure to be completed. Graphical user interface 2800 maypermit the user to pre-load specific calibration files or setup jointpositions based on the procedure being performed and/or the surgeonperforming the procedure. For example, if the user inputs that aprocedure is a laparoscopic cholecystectomy, the system may pre-loadtool types known to be associated with that procedure. Populating thesepre-loaded settings may be achieved by monitoring which tools a usermanually selects for a given procedure. If a given tool is consistentlyselected for a predetermined number of procedures, the system mayautomatically pre-populate that tool the next time the procedure isselected by the user.

In addition, the operator may adjust the vertical and horizontalposition of each robot arm, as shown in FIGS. 27C and 27D above. Asshown in FIG. 28B, to adjust the vertical and/or horizontal position ofthe robot arm that will be or is currently coupled to the retractordevice, the operator may toggle adjustment actuator 2802, and to adjustthe vertical and/or horizontal position of the robot arm that will be oris currently coupled to the endoscope device, the operator may toggleadjustment actuator 2804. In some embodiments, the user may adjust thehorizontal and vertical position of the robot arms by using the robotarm as a force sensitive input device. For example, the robot arm may beconfigured to sense the user’s intention by measuring the force appliedby the user onto the robot arm. If the user applies a force in thepositive horizontal direction, platform may move the robot arm in thatdirection until the user no longer applies a force. A similar approachbe taken for the other directions, e.g., negative horizontal, positivevertical, and negative vertical. As shown in FIG. 28B, graphical userinterface 2800 may indicate whether an error, e.g., fault condition, isdetected by the system during setup or operation of the system, viaerror notification 2810.

As shown in FIG. 28C, graphical user interface 2800 may displayinformation associated with the selected surgical instruments, asdescribed above. For example, graphical user interface 2800 may display,for each instrument to be coupled to each robot arm, the instrumenttype, overall length, distance between the coupler body and theinstrument tip, distance between the center of mass to the instrumenttip, mass, and the preset unlocking force required to unlock theinstrument. As shown in FIG. 28C, graphical user interface 2800 maypermit the operator to select between a high or low unlocking force ofthe surgical instrument. In addition, graphical user interface 2800 maypermit the operator to initiate a surgical instrument calibration, e.g.,for a new surgical instrument that does not already have an associatedcalibration file stored in the system. FIG. 28D illustrates an exemplaryscreen during operation of the system, e.g. during a surgical procedure.As shown in FIG. 28D, graphical user interface 2800 may display thetrocar force and the force being applied to the tip of the surgicalinstrument, e.g., by tissue within the patient’s body.

Referring now to FIG. 29 , an alternative co-manipulation surgical robotsystem is provided. System 2900 may be constructed similar to system 200of FIG. 2 . For example, platform 1400′, base portion 302′, shoulderportion 304′, encoders E1′, E2′, E3′, E5′, E6′, E7′, motor M1′, shoulderjoint 318′, shoulder link 305′, elbow joint 322′, elbow link 310′, wristportion 311′, and coupler interface 400′ for coupling surgicalinstrument SI to the robot arm, may be constructed similar to platform1400, base portion 302, shoulder portion 304, encoders E1, E2, E3, E5,E6, E7, motor M1, shoulder joint 318, shoulder link 305, elbow joint322, elbow link 310, wrist portion 311, and coupler interface 400,respectively. System 2900 differs from system 200 in that system 2900includes motors disposed at the joints of the robot arm. For example,system 2900 may include motor M2′ disposed at elbow joint 318′ and motorM3′ disposed at elbow joint 322′, configured to rotate the associatedlinks to manipulate the robot arm. In addition, encoder E4′ may bepositioned on or adjacent to elbow join 322′.

Some implementations of the systems described herein may be configuredto be controlled or manipulated remotely, e.g., via joystick or othersuitable remote control device, computer vision algorithm, forcemeasuring algorithm, and/or by other means. However, in a preferredembodiment, the systems described herein operate without any telemetry,e.g., the robot arm is not teleoperated via a remote surgeon consoleseparate from the robot arm, but instead the robot arm moves in responseto movement applied to the surgical instrument coupled thereto. Anyrobot-assisted movements applied to the surgical instrument by thesystem, e.g., in the robotic assist mode, are not responsive to userinput received at a remote surgeon console.

FIG. 30A illustrates a top view of coupler 3000 for coupling surgicalinstrument SI to the robot arm, showing coupler body 3002 (also referredto herein as a body) coupled with coupler interface 3001 (also referredto herein as an interface). FIG. 2B illustrates a top view of coupler3000 of FIG. 30A, showing coupler body 3002 decoupled from couplerinterface 3001. As shown in FIGS. 30A and 30B, coupler 3000 may havecoupler body 3002 and coupler interface 3001. Coupler interface 3001 maybe coupled with robotic arm 300 and may be configured such that couplerbody 3002 may be removably coupled with coupler interface 3001. Couplerbody 150 may be coupled with surgical instrument SI at any desired axialposition on surgical instrument SI. Once coupler body 3002 is coupledwith surgical instrument SI, coupler body 3002 and surgical instrumentSI that is coupled with coupler body 3002 may be coupled with couplerinterface 3001. Coupler body 3002 may be configured such that, oncecoupler body 3002 is coupled with surgical instrument SI, surgicalinstrument SI may be at least inhibited (e.g., prevented) from movingaxially or, in some embodiments, moving axially and rotationallyrelative to coupler body 3002. Coupler 3000 may be configured such thatcoupler body 3002 may be at least inhibited (e.g., prevented) frommoving in any axial direction relative to coupler interface 3001. Insome embodiments, coupler 3000 may be configured such that coupler body3002 is free to rotate relative to coupler interface 3001. In thisconfiguration, surgical instrument SI coupled with coupler body 3002 maybe free to rotate relative to coupler interface 3001 that coupler body3002 is coupled with, and may be at least inhibited from (e.g.,prevented from) any axial movement relative to coupler interface 3001that coupler body 3002 is coupled with.

In other embodiments, coupler 300 may be configured such that surgicalinstrument SI may be moved in an axial direction relative to couplerbody 3002 upon the application of at least a threshold force on surgicalinstrument SI relative to coupler body 3002 or upon actuation of arelease or a state change of coupler body 3002. Such actuation may beachieved in some embodiments by, e.g., pressing a button, loosening alocking screw or other connector, moving a dial, or otherwise changingcoupler 3000, coupler body 3002, and/or coupler interface 3001 from asecond, secured state to a first, unsecured state. For example, in someembodiments, surgical instrument SI may be axially repositioned relativeto coupler 3000 by loosening one or more thumbscrews 3010 or otherhand-operated fastener or fastening mechanism such as a clamp in couplerbody 3002, repositioning surgical instrument SI in the desired axialposition, and retightening thumbscrew 3010 or other hand-operatedfastener or fastening mechanism.

As shown in FIG. 30B, coupler interface 3001 may have recess 3003 sizedand shaped to receive coupler body 3002. Recess 3003 may inhibit (e.g.,prevent) an axial movement or, in some embodiments, an axial and arotational movement of coupler body 3002 relative to coupler interface3001 while permitting free rotational movement of coupler body 3002relative to coupler interface 3001. Coupler 3000 may be configured suchthat surgical instrument SI may be at least inhibited (e.g., prevented)from rotational movement relative to coupler 3000. This may be achievedby at least inhibiting (e.g., preventing) the rotational movementbetween surgical instrument SI and coupler 3000, or between coupler body3002 and coupler interface 3001. In some embodiments, a surgical drapemay be pinched or clamped between coupler body 3002 and couplerinterface 3001.

FIG. 30C illustrates an end view of coupler body 3002 and surgicalinstrument SI, showing coupler body 3002 in the first, unsecured or openstate in which surgical instrument SI may be removed and replaced orrepositioned relative to coupler body 3002. FIG. 30D illustrates an endview of coupler body 3002 of FIG. 30C, showing coupler body 3002 in thesecond, secured or closed state in which surgical instrument SI may beat least inhibited (e.g., prevented) from axial movement or, in someembodiments, axial and rotational movement relative to coupler body3002. In some embodiments, coupler body 3002 may have first portion 3004and second portion 3006. In some embodiments, first portion 3004 may berigidly coupled with second portion 3006 via hinge 3005 or shaft orotherwise. In some embodiments, first and second portions 3004, 3006 mayhave a semicircular cut out or recess 3008 therein sized and shaped toreceive surgical instrument SI therein. Fastener 3010 may be used tocouple first portion 3004 with second portion 3006, such as whensurgical instrument SI is positioned in recesses 3008, as shown in FIG.30D. As described above, coupler body 3002 may be configured to at leastsubstantially inhibit (e.g., prevent) an axial movement or, in someembodiments, an axial and a rotational movement of surgical instrumentSI relative to coupler body 3002. Rubber pads, sheets, bumps, O-rings,projections, or other components or features configured to grip anoutside of surgical instrument SI may be used with any of the couplerembodiments disclosed herein. For example, the rubber interface may bepositioned within the recess or recesses of the coupler body, such asrecesses 3008 of first portion 3004 and/or second portion 3006 ofcoupler body 3002 and may be coupled to coupler body 3002. The rubbermay be a silicone rubber or any other suitable type of rubber.

FIGS. 31A to 31D illustrate another embodiment of coupler 3100 that maybe used with any robotic system embodiments disclosed herein to couplean instrument to an end portion of a robot arm. Coupler 3100 may includecoupler body 3101 and coupler interface 3120 that may have a recess ordepression 3190 configured to receive coupler body 3101 therein. Couplerinterface 3120 may be coupled with an end portion of robot arm 300.Coupler 3100 may have coupler body 3101 that removably or nonremovablycouples directly with an end portion of robot arm 300.

As shown in FIG. 31A, coupler body 3101 may have cylindrical bodyportion 3102 having annular flange 3104 projecting away from the surfaceof cylindrical body portion 3102. Body portion 3102 may have opening3106 extending axially through body portion 3102. Opening 3106 may besized and shaped to receive surgical instrument SI therein. Opening 3106may be slightly larger than a diameter or outside size of surgicalinstrument SI. Coupler body 3101 may have one or more deflectable tabs3108 (two being shown), or four or more deflectable tabs 3108 that maybe configured to deflect radially inwardly so that, when tabs 3108 aredeflected radially inwardly, tabs 3108 exert a force on an outsidesurface of surgical instrument SI. Coupler 3100 may be configured suchthat, when coupler body 3101 is positioned within recess 3109 of couplerinterface 3120 and coupler interface 3120 is in a second, closed orsecured state, coupler interface 3120 may exert a force or otherwisedeflect tabs 3108 radially inward so as to grip surgical instrument SIand at least inhibit (e.g., prevent) an axial movement or axial androtational movement of surgical instrument SI relative to coupler body3101. For example, tabs 3108 may have a greater thickness near distalend 3110 of tabs 3108 such that, in a relaxed state or in the first,open state, distal end 3110 of tabs 3108 may project or protrude awayfrom an outside surface of body portion 3102 of coupler body 3101. Inthis configuration, when coupler body 3101 is positioned within recess3109 of coupler interface 3120, moving coupler interface 3120 to thesecond, closed state may cause a force to be applied to distal endportions 3110 of the tabs 3108 to thereby deflect tabs 3108 inwardlyagainst an outside surface of surgical instrument SI.

In some embodiments, recess 3109 may have enlarged portion 3111 sizedand shaped to receive annular flange 3104 therein and to permit arotational movement of flange 3104, while also restricting or at leastinhibiting (e.g., preventing) an axial movement of coupler body 3101 byproviding an axial limit to the movement of annular flange 3104. In thisarrangement, surgical instrument SI may be axially advanced throughopening 3106 of coupler body 3101 to any desired location. Thereafter,surgical instrument SI with coupler body 3101 coupled thereto may bepositioned within recess 3109 of coupler interface 3120. Couplerinterface 3120 may be removably or non-removably coupled with an endportion of robot arm 300 of any of the co-manipulation surgical systemsdisclosed herein.

As shown in FIG. 31C, rubber pads, sheets, bumps, O-rings, projections,or other gripping features 3112 (O-rings being shown) configured to gripan outside of surgical instrument SI may be positioned within couplerbody 3101 to increase a frictional force between surgical instrument SIand coupler body 3101. In some embodiments, one or more tabs 3108 may beconfigured to exert a force on gripping features 3112 when one or moretabs 3108 are deflected inwardly.

As shown in FIG. 31D, coupler interface 3120 may have first portion 3105that may be coupled with second portion 3103. In some embodiments, firstand second portions 3105, 3103 may be rigid and may be coupled to oneanother via mechanical hinge 3107. Alternatively, a living hinge, ashaft, one or more fasteners, or other components or features may beused to couple first and second portions 3105, 3103 together. In someembodiments, second portion 3103 may be flexible and may be configuredto extend over surgical instrument SI and/or a coupler body 3101supported within recess 3109, such as an elastically elongatable or anelastically rigid strap. Additional fasteners, clamps, clasps, or othercomponents or features may be used in conjunction with or in place ofhinge 3107 to securely couple first and second portions 3105, 3103together once coupler body 3101 is received within recess 3109 ofcoupler interface 3120 to securely couple surgical instrument SI withcoupler 3100.

In some embodiments, the coupler may include a coupler body and acoupler interface having a recess configured to receive the couplerbody. The coupler body may have an opening extending axiallytherethrough configured to receive an instrument and an annular flangeextending around an outside surface thereof. The recess in the couplerinterface may have an enlarged portion configured to receive the annularflange and to permit a rotational movement of the flange while at leastinhibiting (e.g., preventing) an axial movement of the coupler body byproviding an axial limit to the movement of the annular flange. Thecoupler interface may be configured to couple with an end portion of arobotic arm.

FIGS. 32A and 32B illustrate coupler body 3200 that may be used with anyrobotic system embodiments disclosed herein to couple an instrument toan end portion of a robot arm. Coupler body 3200 may have any of thecomponents, features, and/or other details of any of the otherembodiments of the coupler body disclosed herein, in any combinationwith any of the components, features, and/or other details of theembodiment of coupler body 3200 shown in FIGS. 32A and 32B. Any of theother embodiments of the coupler body disclosed herein may have any ofthe components, features, and/or other details of coupler body 3200, inany combination with any of the components, features, and/or otherdetails of the other coupler body embodiments disclosed herein.

Coupler body 3200 may have opening 3202 axially therethrough sized andshaped to receive a surgical instrument therein and clamping mechanism3204 configured to reduce an inside diameter of opening 3202 as clampingmechanism 3204 is actuated so as to cause coupler body 3200 to move fromthe first, unsecured or open state as shown in FIG. 32A to the second,secured or closed state as shown in FIG. 32B. In this arrangement,coupler body 3200 may be positioned around an outside surface of thesurgical instrument while coupler body 3200 is in the first, open orunsecured state. Thereafter, clamping mechanism 3204 may be actuated soas to cause coupler body 3200 to secure itself to an outside surface ofa surgical instrument. Then, coupler body 3200 may be coupled with acoupler interface sized and configured to receive and support couplerbody 3200.

FIGS. 33A and 33B illustrate coupler body 3300 that may be used with anyrobotic system embodiments disclosed herein to couple an instrument toan end portion of a robot arm. Coupler body 3300 may have any of thecomponents, features, and/or other details of any of the otherembodiments of the coupler body disclosed herein, in any combinationwith any of the components, features, and/or other details of theembodiment of coupler body 3300. Any of the other embodiments of thecoupler body disclosed herein may have any of the components, features,and/or other details of coupler body 3300, in any combination with anyof the components, features, and/or other details of the other couplerbody embodiments disclosed herein.

Coupler body 3300 may have an opening 3302 axially therethrough sizedand shaped to receive a surgical instrument therethrough and clampingmechanism 3304 having a first and second handle member or tab configuredto reduce an inside diameter of opening 3302 as clamping mechanism 3304is actuated so as to cause coupler body 3300 to move from the first,unsecured or open state as shown in FIG. 33A to the second, secured orclosed state as shown in FIG. 33B. In this arrangement, coupler body3300 may be positioned around an outside surface of the surgicalinstrument while coupler body 3300 is in the first, open or unsecuredstate. Coupler body 3300 may be moved to the first, open or unsecuredstate by squeezing or moving the handles of clamping mechanism 3204together, as shown in FIG. 33A. Thereafter, clamping mechanism 3204 maybe released so as to cause coupler body 3300 to secure itself to anoutside surface of a surgical instrument. Coupler body 3300 may then becoupled with a coupler interface sized and configured to receive andsupport coupler body 3300.

FIGS. 34A to 34C illustrate coupler 3400 that may be used with anyrobotic system embodiments disclosed herein to couple an instrument toan end portion of a robot arm. Coupler 3400 may have any of thecomponents, features, and/or other details of any of the other couplerembodiments disclosed herein, in any combination with any of thecomponents, features, and/or other details of the embodiment of coupler3400. Any of the other coupler embodiments disclosed herein may have anyof the components, features, and/or other details of coupler 3400, inany combination with any of the components, features, and/or otherdetails of the other coupler embodiments disclosed herein.

Coupler 3400 may have one or more coupler bodies 3402 (two being shown)coupled with coupler interface 3404. Coupler bodies 3402 may be slidablyreceived within openings 3406 in coupler interface 3404. Couplerinterface 3404 may have recess 3408 which may have a semicircularcross-sectional shape or other cross-sectional shape that matches ashape of an outside surface of the surgical instrument extending along alength thereof that may be configured to receive an outside surface ofsurgical instrument SI therein. Coupler bodies 3402 may have a curvedend portion 3410 sized and shaped to route or curve at least partiallyaround an outside surface of surgical instrument SI. In thisconfiguration, coupler bodies 3402 when in a second, secured or closedposition as shown in FIG. 34A, may be used to selectively securesurgical instrument SI in recess 3408 or otherwise secure surgicalinstrument SI to coupler interface 3404. Springs or other biasingmechanisms 3412 may be used to bias coupler bodies 3402 in the second,closed or secured position, as shown in FIG. 34A. The user may pushcoupler bodies 3402 in the axial direction indicated by arrow A1 so asto move coupler bodies 3402 from the second, closed or secured positionto the first, open or unsecured position. The force exerted on couplerbodies 3402 should be greater than the spring or biasing force from thespring or biasing mechanisms 3412 coupled with each of coupler bodies3402.

As shown in FIG. 34B, coupler bodies 3402 may have sloped end surface3414. Sloped end surface 3414 may be configured such that a spacebetween coupler end surface 3414 and an adjacent surface of couplerinterface 3404 is greater at a position of coupler end surface 3414 thatis further away from the recess such that, as surgical instrument SI isadvanced laterally toward recess 3408 in coupler interface 3404, anoutside surface of surgical instrument SI may contact end surface 3414of the coupler body and the slope of end surface 3414 of coupler body3400 will cause coupler body 3400 to move from the second, closed orsecured state toward a first, open or unsecured state to permit surgicalinstrument SI to be received within recess 3408. Coupler body 3400 mayhave a spring or other biasing mechanism 3416 configured to bias couplerbody 3400 to the second, closed or secured state or position.

As shown in FIG. 34C, sloped end surface 3414 of any embodiments ofcoupler bodies 3402 may be sloped such that, as surgical instrument SIis advanced in a downward direction relative to end surface 3418 ofcoupler body 3400, such interaction between an outside surface ofsurgical instrument SI and sloping surface 3418 of coupler body 3402 maycause coupler body 3400 to rotate about pivot point 3420 away fromrecess 3408 and permit surgical instrument SI to be received withinrecess 3408.

FIGS. 35A to 35D illustrate coupler 3500 that may be used with anyrobotic system embodiments disclosed herein to couple an instrument toan end portion of a robot arm. Coupler 3500 may have any of thecomponents, features, and/or other details of any of the other couplerembodiments disclosed herein, in any combination with any of thecomponents, features, and/or other details of the embodiment of coupler3500. Any of the other coupler embodiments disclosed herein may have anyof the components, features, and/or other details of the coupler 3500,in any combination with any of the components, features, and/or otherdetails of the other coupler embodiments disclosed herein.

Coupler 3500 may have coupler body 3502 that may be coupled with orengaged with coupler interface 3504. For example, coupler body 3502 maybe slidably received within recess 3506 formed in coupler interface3504. Coupler body 3502 also may have recess 3505 that may have asemicircular cross-sectional shape or other cross-sectional shape thatmatches a shape of an outside surface of the surgical instrumentextending along a length of coupler body 3502 that may be configured toreceive and at least partially surround, or in some embodiments fullysurround, an outside surface of surgical instrument SI at least whencoupler 3500 is in the second state, as shown in FIG. 35B.

Coupler body 3502 may be made from a flexible material, such as rubberincluding neoprene. Coupler body 3502 may have a width that is greaterthan a width of the recess and may be biased toward a planar orgenerally planar shape, as shown in FIG. 35A. Coupler body 3502 may beflexible enough such that, when coupler body 3502 is forced toward adistal surface 3506 a of recess 3506, coupler body 3502 will bend orfold about a middle portion or other portion adjacent to recess 3505.Once coupler body 3502 is fully advanced into recess 3506 of couplerinterface 3504, coupler 3500 may be configured to bias coupler body 3502to remain within the second, secured position within recess 3506. Inthis configuration, to secure surgical instrument SI in coupler 3500, anoperator can advance surgical instrument SI into recess 3505 of couplerbody 3502, and continue to advance surgical instrument SI and/or couplerbody 3502 toward distal surface 3506 a. Some embodiments of coupler 3500may be configured such that, once coupler body 3502 and surgicalinstrument SI have been advanced into recess 3506 of coupler interface3504, surgical instrument SI will be axially and/or rotationally securedto coupler 3500. Thereafter, coupler 3500 may be coupled with an endportion of robot arm 300 such that robot arm 300 may be coupled withsurgical instrument SI. In any embodiments, the recess may have sloped,curved, or otherwise tapered leading edge surfaces 3507 leading into therecess to facilitate the advancement of coupler body 3502 into recess3506 of coupler interface 3504.

As shown in FIG. 35E, surgical drape 800 may be positioned betweensurgical instrument SI and coupler body 3302. In other embodiments,surgical drape 800 may be integrated into coupler body 3052 so thatcoupler body 3502 may form a portion of the surgical drape, as shown inFIG. 35F. Coupler body 3502 may be flexible enough to return to theoriginal shape of coupler body 3502 once coupler body 3502 is removedfrom recess 3506. I n any embodiments disclosed herein, the coupler bodyor other components or features of the coupler can be configured toradially restrain the instrument.

As shown in FIG. 35C, coupler 3500 may be configured such that couplerbody 3502 has a projection 3503 configured to extend into recess 3506 ofcoupler interface 3504 even when coupler body 3502 is in the first, openor unsecured state as shown in FIG. 35C. Projection 3503 may help biascoupler body 3502 to remain engaged with recess 3506 of couplerinterface 3504 even when coupler body 3502 is in the first, open orunsecured state. As shown in FIG. 35D, coupler body 3502 also may haveprotrusions, flanges, handles, tabs, or other projections 3509 at aproximal end portion thereof configured to facilitate gripping andremoval of coupler body 3502 from recess 3506.

In some embodiments, the coupler may include a coupler body made from aflexible material and a coupler interface having a recess configured toreceive the coupler body. The coupler body may have a recess having acurved profile along a length of a first main surface thereof that isconfigured to receive an instrument therein. The coupler body may beflexible enough such that, when the coupler body is forced toward adistal surface of the recess, the coupler body will fold about a portionthereof adjacent to the recess, thereby at least axially and radiallyrestraining the instrument. The coupler body may be flexible enough toreturn to the original shape of coupler body 3502 once the coupler bodyis removed from the recess.

FIG. 36 illustrates coupler 3600 that may be used with any roboticsystem embodiments disclosed herein to couple an instrument to an endportion of a robot arm. Coupler 3600 may have any of the components,features, and/or other details of any of the other coupler embodimentsdisclosed herein, in any combination with any of the components,features, and/or other details of the embodiment of coupler 3600. Any ofthe other coupler embodiments disclosed herein may have any of thecomponents, features, and/or other details of coupler 3600, in anycombination with any of the components, features, and/or other detailsof the other coupler embodiments disclosed herein.

Coupler 3600 may have a coupler body 3602 that may be coupled with orengaged with coupler interface 3604. For example, coupler body 3602 maybe received within recess 3606 formed in coupler interface 3604. Couplerbody 3602 also may have recess 3615 that may have a semicircularcross-sectional shape or other cross-sectional shape that matches ashape of an outside surface of the surgical instrument extending along alength of coupler body 3602 that may be configured to receive and atleast partially surround, or in some embodiments fully surround, anoutside surface of surgical instrument SI at least when coupler 3600 isin the second state.

Coupler body 3202 may be made from a flexible material, such as rubberincluding neoprene. Other embodiments of coupler body 3202 may be madefrom multiple materials, including first layer 3608 made from a flexiblematerial that may have increased gripping such as a rubber and secondlayer 3610 that may be a backing layer or support layer for first layer3608 may be made from a more rigid material, such as plastic, metal, orotherwise. Recess 3615 may be formed in first layer 3608. Recess 3615may be formed in a middle portion of first layer 3608. Some embodimentsof second layer 3610 may have hinge 3612 in or attached to a middleportion thereof. In some embodiments, hinge 3612 may run generallyparallel to recess 3615 formed in first layer 3608 and recess 3606formed in coupler interface 3604. In some embodiments, coupler body 3602may fold or hinge between the first, open state and the second, closedor secured state about surgical instrument SI by folding or hingingabout hinge 3612.

Coupler body 3600 may have a width that is greater than a width ofrecess 3606. Coupler body 3602 may be configured such that, when couplerbody 3602 is forced toward distal surface 3606 a of recess 3606, couplerbody 3602 will bend or fold about hinge 3612 so as to collapse or closeabout surgical instrument SI positioned within recess 3615 of couplerbody 3602 so as to secure surgical instrument SI within coupler body3602 and coupler interface 3604.

Some embodiments of coupler interface 3604 may have one or more rollers3614 (two being shown) at proximal end 3606 b of the recess 3606 formedin coupler interface 3604. The one or more rollers 3614 may facilitatethe movement of coupler body 3602 into recess 3606 by permitting couplerbody 3602 to roll on the rollers as coupler body 3602 is advanced intorecess 3606. Some embodiments of coupler interface 3604 may haveadditional rollers 3616 along the side wall surfaces 3606 c of recess3606 to continue to facilitate the advancement of coupler body 3602 intorecess 3606. In some embodiments, recess 3606 may have a generallyrectangular shape. In other embodiments, recess 3606 may have a taperedor narrowing profile.

Once coupler body 3602 is fully advanced into recess 3606 of couplerinterface 3604, some embodiments of coupler 3600 may be configured tobias coupler body 3602 to remain within the second, secured positionwithin recess 3606. In this configuration, to secure surgical instrumentSI in coupler 3600, an operator may advance surgical instrument SI intorecess 3615 of coupler body 3602, and continue to advance surgicalinstrument SI and/or coupler body 3602 toward distal surface 3606 a ofrecess 3606. Some embodiments of coupler 3600 may be configured suchthat, once coupler body 3602 and surgical instrument SI have beenadvanced into recess 3606 of coupler interface 3604, surgical instrumentSI will be axially and/or rotationally secured to coupler 3600.Thereafter, coupler 3600 may be coupled with an end portion of robot arm300 such that robot arm 300 may be coupled with surgical instrument SI.

FIG. 37 illustrates coupler 3700 that may be used with any roboticsystem embodiments disclosed herein to couple an instrument to an endportion of a robot arm. Coupler 3700 may have any of the components,features, and/or other details of any of the other coupler embodimentsdisclosed herein, in any combination with any of the components,features, and/or other details of the embodiment of coupler 3700. Any ofthe other coupler embodiments disclosed herein may have any of thecomponents, features, and/or other details of coupler 3700, in anycombination with any of the components, features, and/or other detailsof the other coupler embodiments disclosed herein.

Coupler 3700 may have coupler body 3702 that may be coupled with orengaged with coupler interface 3704. Coupler body 3702 may be receivedwithin recess 3796 formed in coupler interface 3704. Coupler body 3702also may have recess 3705 that may have a semicircular cross-sectionalshape or other cross-sectional shape that matches a shape of an outsidesurface of the surgical instrument extending along a length of couplerbody 3702 that may be configured to receive and at least partiallysurround, or in some embodiments fully surround, an outside surface ofsurgical instrument SI at least when coupler 3704 is in the secondstate, as shown in FIG. 37 .

Coupler body 3702 may be made from multiple materials, including firstlayer 3710 made from a flexible material that may have increasedgripping such as a rubber and second layer 3712 that may be a backinglayer or support layer for first layer 3710 may be made from a morerigid material, such as plastic, metal, or otherwise. Recess 3705 may beformed in first layer 3710. In some embodiments, recess 3705 may beformed in a middle portion of first layer 3710. Some embodiments ofsecond layer 3712 may have hinge 3714 in or attached to a middle portionthereof. In some embodiments, hinge 3714 may run generally parallel torecess 3705 formed in first layer 3710 and recess 3706 formed in couplerinterface 3704. In some embodiments, coupler body 3702 may fold or hingebetween the first, open state and the second, closed or secured stateabout surgical instrument SI by folding or hinging about hinge 3714.

Coupler body 3702 may have a width that is greater than a width ofrecess 3706. Coupler body 3702 may be configured such that, when couplerbody 3702 is forced toward a distal surface 3706 a of the recess 3706,coupler body 3702 will bend or fold about hinge 3714 so as to collapseor close about surgical instrument SI positioned within recess 3705 ofcoupler body 3702 so as to secure surgical instrument SI within couplerbody 3702 and coupler interface 3704. In some embodiments, second layer3712 may have wings or tabs 3716 that may be used to facilitate removalof coupler body 3702 from recess 3706. Tabs 3716 may be formed suchthat, when coupler body 3702 is in the second position, as shown in FIG.37 , tabs 3716 may be spaced apart from first surface 3704 a (which canbe an upper surface when coupler interface 3704 is positioned as shownin FIG. 37 ) such that a gap or space 3720 exists between tabs 3716 andupper surface 3704 a of coupler interface 3704. Space 3720 may be largeenough to permit tabs 3716 to move toward first surface 3704 a when aforce is applied to tabs 3716 in the direction of first surface 3704 a.As tabs 3716 are deflected toward first surface 3704 a, such movement oftabs 3716 may force a remainder of coupler body 3702 to move away from adistal surface 3706 a of recess 3706, thereby allowing coupler body 3704to be removed from recess 3706.

Some embodiments of coupler interface 3704 may have one or more rollers3717 (two being shown) at proximal end 3706 b of recess 3706 formed incoupler interface 3704. The one or more rollers 3717 may facilitate themovement of coupler body 3702 into recess 3706 by permitting couplerbody 3702 to roll on the rollers as coupler body 3702 is advanced intorecess 3706. Some embodiments of coupler interface 3704 may haveadditional rollers 3718 along the side wall surfaces 3706 c of recess4706 to continue to facilitate the advancement of coupler body 3702 intorecess 3706.

Once coupler body 3702 is fully advanced into recess 3706 of couplerinterface 3704, some embodiments of coupler 3700 may be configured tobias coupler body 3702 to remain within the second, secured positionwithin recess 3706. In this configuration, to secure surgical instrumentSI in coupler 3700, an operator may advance surgical instrument SI intorecess 3705 of coupler body 3703, and continue to advance surgicalinstrument SI and/or coupler body 3702 toward distal surface 3706 a ofrecess 3706. Some embodiments of coupler 3700 may be configured suchthat, once coupler body 3702 and surgical instrument SI have beenadvanced into recess 3706 of coupler interface 3704, surgical instrumentSI will be axially and/or rotationally secured to coupler 3700.Thereafter, coupler 3700 may be coupled with an end portion of robot arm300 such that robot arm 300 may be coupled with surgical instrument SI.

FIGS. 38A and 38B illustrate coupler 3800 that may be used with anyrobotic system embodiments disclosed herein to couple an instrument toan end portion of a robot arm. Coupler 3800 may have any of thecomponents, features, and/or other details of any of the other couplerembodiments disclosed herein, in any combination with any of thecomponents, features, and/or other details of the embodiment of coupler3800. Any of the other coupler embodiments disclosed herein may have anyof the components, features, and/or other details of coupler 3800, inany combination with any of the components, features, and/or otherdetails of the other coupler embodiments disclosed herein.

Coupler 3800 may have coupler body 3802 that may be coupled with orengaged with a coupler interface (not shown) or may be coupled with orengaged with a robot arm without the presence of a coupler interface(e.g., the coupler body of any embodiments disclosed herein can directlyengage or interface with an end portion of robot arm 300). Coupler body3802 may have first portion 3804 and second portion 3806 coupled withfirst portion 3804. In some embodiments, first portion 3804 may behingedly or rotatably coupled with second portion 3806. For example,coupler body 3802 may have a hinge or joint 3810 that may couple firstand second portions 3804, 3806 together.

In some embodiments, first portion 3804 of coupler body 3802 may haveproximal portion 3804 a and distal portion 3804 b that is integrallyformed with or coupled with proximal portion 3804 a. First portion 3804of coupler body 3802 may have recess 3812 and second portion 3806 ofcoupler body 3820 may have recess 3814, each of which can have asemicircular cross-sectional shape or other cross-sectional shape thatmatches a shape of an outside surface of the surgical instrumentextending along a length of coupler body 3802 that may be configured toreceive and at least partially surround, or in some embodiments fullysurround, an outside surface of surgical instrument SI at least whencoupler 3800 is in the second state. The second state of coupler body3802 is shown FIG. 38B. In some embodiments, second portion 3806 may besimilarly situated and may be a mirror copy of first portion 3804, withproximal portion 3806 a and distal portion 3806 b that is integrallyformed with or coupled with the proximal portion 3806 a.

Some embodiments of coupler 3800 may be configured to be bistable inthat the coupler 3800 will be biased toward either the first, open orunsecured state or the second, closed or secured state and is unstablein any position or state except the first and second states. In thefirst state, distal portion 3804 b of first portion 3804 of coupler 3800is in contact with the distal portion 3806 b of second portion 3806 ofcoupler 3800 and proximal portion 3804 a of first portion 3804 ofcoupler 3800 is rotated away and spaced apart from proximal portion 3806a of second portion 3806 of coupler 3800. In the first, open orunsecured state, surgical instrument SI may be loaded into or removedfrom coupler 3800. In the second state, proximal portion 3804 a of firstportion 3804 of coupler 3800 is in contact with proximal portion 3806 aof second portion 3806 of coupler 3800 and distal portion 3804 b offirst portion 3804 of coupler 3800 is rotated away and spaced apart fromdistal portion 3806 b of second portion 3806 of coupler 3800. In thesecond, closed or secured state, surgical instrument SI loaded intocoupler 3800 may be secured or supported by coupler 3800 such thatsurgical instrument SI may be at least inhibited (e.g., prevented) froman axial movement or, in some embodiments, an axial and a rotationalmovement relative to the coupler 3800.

In this configuration, when coupler 3800 is in the first, open state asshown in 38A, after positioning surgical instrument SI in either recess3812 with recess 3814, the operator may change coupler 3800 to thesecond, closed state by pinching or moving the proximal portion 3804 aof first portion 3804 toward proximal portion 3806 a of second portion3806, such as by exerting a force on proximal portions 3804 a, 3806 a offirst and second portions 3804, 3806 along the directions A3 and A4, asshown in FIG. 38A (e.g., by squeezing the proximal portions 3804 a, 3806a of first and second portions 3804, 3806 together). When coupler 3800is in the second, closed state as shown in FIG. 38B, the operator maychange coupler 3800 to the first, open state by pinching or movingdistal portion 3804 b of first portion 3804 toward distal portion 3806 bof second portion 3806, such as by exerting a force on distal portions3804 b, 3806 b of first and second portions 3804, 3806 along thedirections A5 and A6, as shown in FIG. 38B (e.g., by squeezing distalportions 3804 b, 3806 b of first and second portions 3804, 3806together).

FIGS. 39A and 39B illustrate coupler 3900 that may be used with anyrobotic system embodiments disclosed herein to couple an instrument toan end portion of a robot arm. Coupler 3900 may have any of thecomponents, features, and/or other details of any of the other couplerembodiments disclosed herein, in any combination with any of thecomponents, features, and/or other details of the embodiment of coupler3900. Any of the other coupler embodiments disclosed herein may have anyof the components, features, and/or other details of the coupler 3900,in any combination with any of the components, features, and/or otherdetails of the other coupler embodiments disclosed herein.

Coupler 3900 may have a coupler body 3902 that may be coupled with orengaged with a coupler interface (not shown) or may be coupled with orengaged with a robotic arm without the presence of a coupler interface.Coupler body 3902 may have one or more projections 3903 (two beingshown) that may be used to center or position coupler body 3902 relativeto the coupler interface. For example, projections 3903 may be conicalprojections configured to engage with depressions or openings in thecoupler interface to align coupler body 3902 with the coupler interface.In some embodiments, the coupler interface may have an equal number or adifferent number of depressions or openings as compared to the number ofprojections 3903. In other embodiments, projections 3903 may becylindrically shaped. In some embodiments, coupler body 3902 may havethree or more projections 3903.

Coupler body 3902 may have first tab 3904 hingedly or rotatably coupledwith coupler body 3902 and second tab 3906 hingedly or rotatably coupledwith coupler body 3902. For example, coupler body 3902 may have a firsthinge or joint 3910 that may couple first tab 3904 with coupler body3902 and a second hinge or joint 3911 that may couple second tab 3906with coupler body 3902. First tab 3904 may have proximal end portion3904 a and distal end portion 3904 b, as shown in FIG. 39B. Second tab3906 may have proximal end portion 4906 a and distal end portion 4906 b.

Coupler body 3902 may have recess 3914 formed therein, first tab 3904may have recess 3916 formed in a distal end portion thereof and secondtab 3906 may have recess 3918 formed in a distal end portion thereof,each of which may have a semicircular cross-sectional shape or othercross-sectional shape that, all together, may match a shape of anoutside surface of surgical instrument SI extending along a length ofcoupler body 3902, first tab 3904, and second tab 3906 and that may beconfigured to receive and at least partially surround, or in someembodiments fully surround, an outside surface of surgical instrument SIat least when coupler 3900 is in the second state. The second state ofcoupler body 3902 is shown in FIG. 39B. In some embodiments, second tab3906 may be similarly situated and may be a mirror copy of first tab3904.

Some embodiments of coupler 3900 may be biased toward the second state,using springs or other torsional biasing elements. An operator mayovercome the bias or otherwise move coupler body 3902 from the secondstate as shown in FIG. 39B to the first state as shown in FIG. 39A bysqueezing together or toward one another proximal end portions 3904 a,3906 a of first and second tabs 3904, 3906. In the first state, theoperator may remove surgical instrument SI from coupler 3900. To supporta surgical instrument SI in coupler 3900, while coupler 3900 is in thefirst, open state, the operator may position surgical instrument SI incontact with or near recess 3914 and release the force that was appliedto first and second tab 3904, 3906 or otherwise relax first and secondtab 3904, 3906 and allow first and second tabs 3904, 3906 to return tothe relaxed position of first and second tabs 3904, 3906.

FIGS. 40 to 43 illustrate additional couplers 4000, 4100, 4200, 4300.Couplers 4000, 4100, 4200, 4300 may have any of the components,features, and/or other details of any of the other coupler embodimentsdisclosed herein, in any combination with any of the components,features, and/or other details of the embodiment of couplers 4000, 4100,4200, 4300. Any of the other coupler embodiments disclosed herein mayhave any of the components, features, and/or other details of couplers4000, 4100, 4200, 4300 in any combination with any of the components,features, and/or other details of the other coupler embodimentsdisclosed herein.

As shown in FIG. 40 , coupler 4000 may have first body portion 4002 andsecond body portion 4004 that may be slidably coupled with or engagedwith first body portion 4002. Coupler 4000 may have a recess or opening4006 that may be enlarged and may be configured to receive surgicalinstrument SI therein when second body portion 4004 is moved towardfirst body portion 4002. A spring or other biasing mechanism 4008 may beused to bias coupler 4000 toward the second, closed or secured state sothat, when an operator releases first and second body portions 4002,4004, coupler 4000 may exert a force on a surgical instrument to securethe surgical instrument therein. Some embodiments of coupler 400 may befigured to axially restrain a surgical instrument therein, but to permita rotation of the surgical instrument. Coupler 4000 may be coupled witha coupler interface or directly to an end portion of robot arm 300.

As shown in FIG. 41 , coupler 4100 may have first body portion 4102 andsecond body portion 4104 that may be slidably coupled with or engagedwith first body portion 4102. Coupler 4100 may have a recess or opening4106 that may be enlarged and may be configured to receive surgicalinstrument SI therein when second body portion 4104 is moved towardfirst body portion 4102. Spring 4108 or other biasing mechanism may beused to bias coupler 4100 toward the second, closed or secured state sothat, when an operator releases first and second body portions 4102,4104, coupler 4100 may exert a force on a surgical instrument to securethe surgical instrument therein. Some embodiments of coupler 4100 may befigured to axially restrain a surgical instrument therein, but to permita rotation of the surgical instrument. Coupler 4100 may be coupled witha coupler interface or directly to an end portion of robot arm 300.

As shown in FIG. 42 , coupler 4200 may have first body portion 4202having proximal end portion 4202 a and distal end portion 4202 b andsecond body portion 4204 having proximal end portion 4204 a and distalend portion 4204 b that may be rotatably coupled with or engaged withfirst body portion 4202 about an axis or shaft 4207. Coupler 4200 mayhave a recess or opening 4206 formed in distal end portions 4202 b, 4204b that may be enlarged and may be configured to receive surgicalinstrument SI therein when distal end portion 4204 b of second bodyportion 4204 is rotated away from distal end portion 4202 b of firstbody portion 4202. Spring 4208 or other biasing mechanism may be used tobias coupler 4200 toward the second, closed or secured state so that,when an operator releases first and second body portions 4202, 4204,coupler 4200 may exert a force on a surgical instrument to secure thesurgical instrument therein. Some embodiments of coupler 4200 may befigured to axially restrain a surgical instrument therein, but to permita rotation of the surgical instrument. Coupler 4200 may be coupled witha coupler interface or directly to an end portion of robot arm 300.

As shown in FIG. 43 , coupler 4300 may be configured to engage with acoupler interface or the distal end portion 4301 of a robot arm. Coupler4300 may be constructed similar to coupler 4200, with similar componentshaving like-prime reference numerals. Coupler 4300 differs from coupler4200 in that coupler 4300 may have projections 4302 extending inwardlyfrom an inner surface of proximal end portion 4202 a′ of first bodyportion 4202′ and an inner surface of proximal end portion 4204 a′ ofsecond body portion 4204′ that may be received within recesses 4304formed in distal end portion 4301 of the robot arm when coupler 4300 isin the second, closed state.

Referring now to FIGS. 44A to 50B, another exemplary surgical instrumentcoupling mechanism is provided. FIGS. 44A to 44C illustrate couplingmechanism 4400 positioned at the distal end of distal wrist link 316 ofwrist portion 311 of robot arm 300. Coupling mechanism 4400 may includecoupler interface 4500 at the distal end of the distal-most link of therobot arm (illustratively, link 316), and coupler body 4600, which maybe configured to be removably coupled to a surgical instrument and tocoupler interface 4500, such that a sterile drape may be placed betweencoupler interface 4500 and coupler body 4600. Accordingly, coupler body4600 may be disposable, or alternatively, sterilizeable between surgicalprocedures. Moreover, coupling mechanism 4400 may be operatively coupledto one or more sensors for detecting when coupler body 4600 is coupledto coupler interface 4500, and when a surgical instrument is coupled tocoupler body 4600 when coupler body 4600 is coupled to coupler interface4500, as well as the type/size of the surgical instrument coupled tocoupler body 4600, as described in further detail below.

FIGS. 45A to 45C illustrate coupler interface 4500 at the distal end oflink 316 of the robot arm. As shown in FIGS. 45A and 45B, couplerinterface 4500 may include protrusion 4504 extending from flat portion4502. Flat portion 4502 may have an outer diameter that coincides withthe outer diameter of link 316. Protrusion 4504 may have a non-circularprofile, which corresponds to the geometry of groove 4605 of couplerbody 4600, as described in further detail below. Moreover, protrusion4504 may include one or more locking portions 4506 disposed on the outersurface of the sidewall of protrusion 4504. For example, lockingportions 4506 may be indentations/grooves extending along the outersurface of protrusion 4504, and sized and shaped to engage with lockingarms 4606 of coupler body 4600, as described in further detail below,for securing coupler body 4600 to coupler interface 4500, and forsecuring the sterile drape between coupler body 4600 and couplerinterface 4500.

Preferably, protrusion 4504 includes at least a pair of locking portions4506. For example, as shown in FIG. 45A, protrusion 4504 may have aprofile defined by two straight sides connected by two curved sides,such that a first pair of locking portion 4506 may be disposed on aportion of each of the curved sides adjacent to a first straight side ofprotrusion 4504, and a second pair of locking portion 4506 may bedisposed on a portion of each of the curved sides adjacent to the otherstraight side of protrusion 4504. By having two pairs of lockingportions 4506, coupler body 4600 may be securely coupled to couplerinterface 4500 in two orientations.

As shown in FIG. 45C, coupler interface 4500 may include extendedportion 4508 configured to be inserted within link 316. Moreover,coupler interface 4500 may include a metal rod, e.g., ferrous rod 4510,extending through coupler interface 4500 from protrusion 5404 throughextended portion 4508. Link 316 may include one or more sensors, e.g.,Hall effect sensors 4514, configured to detect a magnetic field inducedin ferrous rod 4510. Preferably, link 316 includes at least two Halleffect sensors to provide redundancy for more accurate magnetic fieldmeasurements. As shown in FIG. 45C, Hall effect sensors 4514 may bepositioned adjacent to a proximal end of ferrous rod 4510. In addition,coupler interface 4500 may include repulsion magnet 4512 disposed withinprotrusion 4504 adjacent to the distal end of ferrous rod 4512.Repulsion magnet 4512 is configured to apply a magnetic force to amagnet slidably disposed within coupler body 4600 to facilitatedetermination of when coupler body 4600 is coupled to coupler interface4500 and no surgical instrument is coupled to coupler body 4600, e.g.,by causing the magnet to move to a position within coupler body 4600with a maximum distance from ferrous rod 4510, and/or to facilitatecoupling of the surgical instrument to coupler body 4600, as describedin further detail below. Moreover, as described above, robot arm 300 mayinclude one or more encoders E7 for measuring angulation of betweenmiddle wrist link 314 and distal wrist link 316 may be disposed on oradjacent to joint 328, e.g., within link 316. For example, encoders E7may include two or more encoders positioned circumferentially aroundextended portion 4508 of coupler interface 4500.

Referring now to 46A to 46E, coupler body 4600 is provided. Coupler body4600 may be configured to be removably coupled to a surgical instrumenthaving a predefined shaft diameter, e.g., a 10 mm surgical instrument.Coupler body 4600 is preferably designed to be locked to the distal endof the robot arm with a sterile drape therebetween such that the robotarm remains covered and sterile throughout a procedure. Further, couplerbody 4600 also has a separate portion for locking to a surgicalinstrument (e.g., a commercially available laparoscopic instrument) topermit the clinician to perform the surgeries with the robot arm(s) asdescribed herein. As shown in FIGS. 46A to 46C, coupler body 4600 mayinclude coupler interface connection portion 4604 and surgicalinstrument connection portion 4602. Coupler interface connection portion4604 and surgical instrument connection portion 4602 may generally havean outer diameter that coincides with the outer diameters of flatportion 4502 of coupler interface 4504 and link 316. As shown in FIG.46C, coupler interface connection portion 4604 may include groove 4605extending inward from a bottom surface of coupler body 4600. Groove 4605may have a geometry that corresponds with the profile shape ofprotrusion 4504 of coupler interface 4500, such that protrusion 4504 maybe received by groove 4605. Accordingly, the geometry of groove 4605 mayinclude two straight sides connected by two curved sides. The steriledrape may be positioned between protrusion 4504 and groove 4605 whenprotrusion 4504 is disposed within groove 4605. Preferably, the profileof protrusion 4504 and the corresponding geometry of groove 4605 aresymmetrical such that protrusion 4504 may be received by groove 4605 inat least two orientations. Moreover, the profile of protrusion 4504 andthe corresponding geometry of groove 4605 may guide the coupling ofcoupler body 4600 to coupler interface 4500 by the user.

Additionally, coupler interface connection portion 4604 may include apair of locking arms 4606 configured to facilitate securing of couplerbody 4600 to coupler interface 4500 when protrusion 4504 is disposedwithin groove 4605. Each of locking arms 4606 may include handle portion4610 sized and shaped to be actuated by the user’s fingers, andconnection portion 4608 sized and shaped to engage with locking portions4506 of protrusion 4504. For example, connection portion 4608 may have atapered profile for securely engaging with locking portion 4506. Lockingarms 4606 may be pivotally coupled to coupler interface connectionportion 4604, such that locking arms 4606 may be transitionable betweenan unlocked state and a locked state. Moreover, locking arms 4606 may bepivotally coupled to coupler interface connection portion 4604 via aspring, e.g., a torsion spring, an extension spring, a compressionspring, etc., such that locking arms 4606 are biased toward the lockedstate. Accordingly, handle 4610 may be actuated to transition lockingarms 4606 from the locked state to the unlocked state.

As shown in FIGS. 46A to 46C, surgical instrument connection portion4602 may include opening 4616 extending therethrough, sized and shapedto receive the shaft of a surgical instrument. For example, opening 4616may be sized and shaped to receive a 10 mm surgical instrument shaft.Opening 4616 may be defined by a channel extending downward from anupper surface of surgical instrument connection portion 4602 such that asurgical instrument may be inserted into opening 4616 via the channel.As shown in FIGS. 46A and 46B, the upper surface of surgical instrumentconnection portion 4602 may include tapered portions 4614 that angledownward towards opening 4616, thereby defining the channel into opening4616. Accordingly, tapered portions 4614 ensure that the shaft of thesurgical instrument is properly inserted into opening 4616 in one of twoorientations by rotating coupler body 4600 and accordingly distal wristlink 316 to align with the longitudinal axis of the surgical instrumentin one of two orientations. For example, tapered portions 4614 mayfacilitate in “self-alignment” of the distal end of robot arm 300, e.g.,by causing distal wrist link 316, which is coupled to coupler body 4600via coupler interface 4500, to automatically rotate relative to middlewrist link 314 about axis Q7 at passive joint 328 as the instrumentshaft is guided down tapered portions 4614, such that the longitudinalaxis of opening 4616 aligns with the longitudinal axis of the surgicalinstrument. Accordingly, the user does not need to align the instrumentshaft to opening 4616, but rather, opening 4616 rotates via rotation ofcoupler body 4600 and distal wrist link 316 relative to middle wristlink 314 to align with the longitudinal axis of the instrument shaft.

In addition, surgical instrument connection portion 4602 may includeclamp 4618 pivotally coupled to surgical instrument connection portion4602 about axis 4612, such that clamp 4618 may be transitionable betweenan unlocked state and a locked state. Moreover, clamp 4618 may bepivotally coupled to surgical instrument connection portion 4602 via atorsion spring, such that clamp 4618 is biased toward the locked state.Clamp 4618 may include locking portion 4620 configured to secure thesurgical instrument within opening 4616 when clap 4618 is in its lockedstate. For example, a lower surface of locking portion 4620 may definethe upper surface of opening 4616 when clamp 4618 is in its lockedstate, such that locking portion 4620 prevents upward movement of thesurgical instrument when the surgical instrument is positioned withinopening 4616 and clamp 4618 is in its locked state.

The upper surface of locking portion 4620 may be tapered to facilitateguidance of the surgical instrument into opening 4616 along with taperedportions 4614. For example, the upper surface of locking portion 4620may have a tapered angle that more or less coincides with the taperedangles of tapered portions 4614. Accordingly, in some embodiments, thetapered angle of locking portion 4620 may be alone sufficient to permita surgical instrument to be inserted into opening 4616, such thatinsertion of the surgical instrument towards opening 4616 applies aforce against the tapered upper surface of locking portion 4620, therebycausing clamp 4618 to rotate about axis 4612 from the locked state tothe unlocked state to permit the surgical instrument to be received byopening 4616. Clamp 4618 further may include handle 4622 sized andshaped to be actuated by the user’s fingers to transition clamp 4618from the locked state to the unlocked state. For example, handle 4622may be actuated to transition clamp 4618 to the unlocked state forinsertion of the surgical instrument into opening 4616, and/or forremoval of the surgical instrument from opening 4616.

Moreover, coupler body 4600 further may include switch 4624 pivotallycoupled to surgical instrument connection portion 4602, and configuredto facilitate securement of the surgical instrument within opening 4616.For example, switch 4624 may include one or more surgical instrumentengagement portions 4626, each having a geometry that corresponds withthe outer diameter of the shaft of the surgical instrument to beinserted within opening 4616. In addition, switch 4624 may includehandle portion 4628 sized and shaped to be actuated by the user’sfingers to transition switch 4624 between an unlocked state and a lockedstate where surgical instrument engagement portion 4626 engages with thesurgical instrument shaft within opening 4616 and applies a frictionforce to the surgical instrument shaft.

In some embodiments, switch 4624 may include two surgical instrumentengagement portions 4626, one on each side of switch 4624, such thatswitch 4624 may be rotated from its unlocked state in either directionto transition to its locked state where one of the surgical instrumentengagement portions will engage with the surgical instrument withinopening 4616. Accordingly, in its locked state, surgical instrumentengagement portion 4626 further defines opening 4616. Surgicalinstrument engagement portion 4626 may have a coefficient of friction,such that when the surgical instrument is disposed within opening 4616and switch 4624 is in its locked state, surgical instrument engagementportion 4626 applies a friction force against the surgical instrumentthat prevents longitudinal movement of the surgical instrument relativeto coupler body 4600, while permitting rotational movement of thesurgical instrument within opening 4616. Accordingly, when the surgicalinstrument is disposed within opening 4616, switch 4624 may be actuatedto its unlocked state to permit the user to readjust/move the surgicalinstrument longitudinally relative to coupler body 4600 within opening4616, and back to its locked state to prevent longitudinal movement ofthe surgical instrument relative to coupler body 4600. Preferably, bothswitch 4624 and clamp 4618 must be in their unlocked states to permitremoval of the surgical instrument from coupler body 4600.

Coupler body 4600 further may include holder 4630 disposed withinsurgical instrument connection portion 4602, as shown in FIG. 46E.Referring now to FIG. 46D, holder 4630 is provided. Holder 4630 isconfigured to be slidably disposed within channel 4603 within surgicalinstrument connection portion 4602, e.g., toward or away from couplerinterface connection portion 4604. Moreover, holder 4630 is configuredto hold magnet 4640, and provide friction to the surgical instrumentdisposed within opening 4616 to prevent longitudinal movement of thesurgical instrument within opening 4616 when switch 4624 is in itslocked state. As shown in FIG. 46D, holder 4630 may include a contactsurface, e.g., friction pad 4632, disposed on an upper portion of cradle4634, such that the upper surface of friction pad 4632 defines the lowersurface of opening 4616. The upper surface of friction pad 4632 may havea curved profile, which may coincide with the curvature of the surgicalinstrument. Friction pad 4632 may have a coefficient of friction, suchthat when the surgical instrument is disposed within opening 4616 andswitch 4624 is in its locked state, friction pad 4632 applies a frictionforce against the surgical instrument that prevents longitudinalmovement of the surgical instrument relative to coupler body 4600, whilepermitting rotational movement of the surgical instrument within opening4616. Although FIG. 46D shows friction pad 4632 as a single piece,friction pad 4632 may be formed of multiple pieces configured to contactthe surgical instrument within opening 4616, or alternatively, may bewrapped around the upper surface of holder 4630 or otherwise integratedwith holder 4630. When switch 4624 is moved to its unlocked state, thefriction force of friction pad 4632 may not be sufficient to preventlongitudinal movement of the surgical instrument relative to couplerbody 4600.

As shown in FIG. 46D, holder 4634 may include two cradles 4634 extendingbetween friction pad 4632 and magnet harness 4638. Accordingly, frictionpad 4632 may extend longitudinally between cradles 4632. Friction padmay be supported via a support extending between cradles 4634. Moreover,each cradle 4634 of holder 4630 may include channel 4636 extendingwithin cradle 4634 in a direction from magnet harness 4638 towardsfriction pad 4632. Channels 4636 may be sized and shaped to slidablyreceive a longitudinally extending rod therethrough, such that thelongitudinally extending rod extends along axis 4612 between channels4636. Clamp 4618 may be pivotally coupled to the longitudinallyextending rod, such that clamp 4618 may rotate about axis 4612, asdescribed above. Axis 4612 may be fixed relative to surgical instrumentconnection portion 4602, such that holder 4630 may move toward/away fromcoupler interface connection portion 4604 via movement of channel 4636along the longitudinally extending rod.

As shown in FIG. 46D, magnet harness 4638 may extend between cradles4634, and may be configured to securely hold magnet 4640. Magnet harness4638 may have a geometry that corresponds with the geometry of channel4603, such that magnet harness may slidably move within channel 4603,but cannot move beyond the lower or upper surface of channel 4603.Magnet 4640 may have a magnetic force such that when coupler body 4600is coupled to coupler interface 4500, magnet 4640 induces a magneticfield in ferrous rod 4510, which may be detected by Hall effectorsensors 4514. Accordingly, the strength of the magnetic field induced inferrous rod 4510 will be proportional to the distance between magnet4640 and ferrous rod 4510, such that the magnetic field detected by Halleffector sensors 4514 may be indicative of the position of magnet 4640,and accordingly holder 4630, within coupler body 4600. Similarly, whenno magnetic field is induced in ferrous rod 4510 via magnet 4640, Halleffector sensors 4514 may detect that coupler body 4600 is not coupledto coupler interface 4500. Moreover, repulsion magnet 4512 of couplerinterface 4500 may have a magnetic force such that when coupler body4600 is coupled to coupler interface 4500, repulsion magnet 4512 appliesa magnetic force to magnet 4640 to thereby cause magnet 4640, andaccordingly holder 4630, to move away from coupler interface connectionportion 4604. The position of holder 4600 relative to coupler body 4600may be indicative of whether a surgical instrument is or is not coupledto coupler body 4600 when coupler body 4600 is coupled to couplerinterface 4500.

For example, FIG. 47A illustrates coupler body 4600 when coupler body4600 is not coupled to coupler interface 4500. As shown in FIG. 47A,without repulsion magnet 4512 of coupler interface 4500 within thevicinity of magnet 4640, no magnetic force will be applied to magnet4640 to cause displacement of holder 4630, e.g., toward opening 4616.Accordingly, holder 4630 may be in a neutral position, e.g., towardscoupler interface connection portion 4604 due to gravity. As shown inFIG. 47B, when coupler body 4600 is coupled to coupler interface 4500and no surgical instrument is coupled to coupler body 4600, repulsionmagnet 4512 may apply a magnetic force to magnet 4640, thereby causingmagnet 4640, and accordingly holder 4630, to move towards opening 4616and away from coupler interface 4500 within channel 4603, e.g., to aposition within coupler body 4600 with a maximum distance from ferrousrod 4510. For example, when coupler body 4600 is coupled to couplerinterface 4500, the magnetic force of repulsion magnet 4512 may causemagnet harness 4638 of holder 4630 to be positioned adjacent to theupper surface of channel 4603, as shown in FIG. 47B. Thus, when couplerbody 4600 is coupled to coupler interface 4500, friction pad 4632 may becloser to locking portion 4620 of clamp 4618, thereby reducing the sizeof opening 4616.

Moreover, the magnetic field induced in ferrous rod 4510 by magnet 4640when magnet 4640 is in the position within channel 4603 farthest awayfrom ferrous rod 4510 responsive to the magnetic force of repulsionmagnet 4512 when coupler body 4600 is coupled to coupler interface 4500and no instrument is coupled to coupler body 4600, may provide a cleansignal that may be detected by Hall effect sensors 4514, indicative ofcoupler body 4600 being coupled to coupler interface 4500 without asurgical instrument attached thereto. Accordingly, the system maydetermine that coupler body 4600 is coupled to coupler interface 4500with no surgical instrument coupled to coupler body 4600, based on thestrength of the magnetic field induced in ferrous rod 4510 by magnet4640, e.g., when magnet 4640 is a maximum distance from ferrous rod 4510within coupler body 4600. In some embodiments, coupler interface 4500may not have repulsion magnet 4512, such that holder 4630 may be biasedtoward opening 4616 via a compression spring that applies a spring forceto holder 4630, e.g., via magnet harness 4638.

In some embodiments, the magnetic force and/or the polarity of magnet4640 may be selected based on the coupler body size, e.g., the size ofthe surgical instrument that may be coupled to the coupler body.Accordingly, different sized coupler bodies may have differentpredefined magnetic forces based on the magnet used therein. Thus, uponcoupling of the coupler body to the coupler interface, the magneticfield induced in the ferrous rod responsive to the predefined magneticforce of the magnet within the coupler body may be indicative of thesurgical instrument size even before the surgical instrument is coupledto the coupler body.

FIGS. 48A and 48B illustrate the coupling of coupler interfaceconnection portion 4604 of coupler body 4600 to coupler interface 4500.As described above, locking arms 4606 may be biased toward its lockedstate where connection portions 4608 are moved radially inward, suchthat at least a portion of connection portions 4608 are positionedwithin groove 4605 of coupler interface connection portion 4604. Priorto coupling coupler body 4600 to coupler interface 4500, a sterile drapemay be positioned between coupler body 4600 an coupler interface 4500,such that the sterile drape may be draped over robot arm 300, asdescribed above. Next, the user may apply a force to handle portions4610 of locking arms 4606, e.g., pinch handle portions 4610 toward eachother, to thereby cause connection portions 4608 to move away from eachother towards the unlocked state and out of groove 4605, and provideclearance for protrusion 4504 to be received within groove 4605, asshown in FIG. 48A.

Accordingly, when locking arms 4606 are in their unlocked state, couplerbody 4600 may be coupled to coupler interface 4500 such that protrusion4504 is disposed within groove 4605. Once protrusion 4504 is disposedwithin groove 4605, the user may release handle portions 4610, such thatlocking arms 4606 move back towards their locked state and connectionportion 4608 engages with locking portion 4506 of protrusion 4506, asshown in FIG. 48B. Accordingly, the engagement of connection portion4608 and locking portion 4506 due to the corresponding geometries ofconnection portion 4608 and locking portion 4506 may prevent movementbetween coupler body 4600 and coupler interface 4500, to therebysecurely couple coupler body 4600 to coupler interface 4500.

FIG. 49 illustrates when a surgical instrument is coupled to couplerbody 4600 when coupler body 4600 is coupled to coupler interface 4500.As described above, coupler body 4600 may be configured to be coupledto, e.g., a 10 mm surgical instrument such as surgical instrument 10described above with regard to FIGS. 7A to 7D. Accordingly, opening4616, which is at least partially defined by the lower surface oflocking portion 4620 of clamp 4618 and the moveable upper surface offriction pad 4632 of holder 4630, may be sized and shaped to accommodateshaft 10 a of surgical instrument 10. As shown in FIG. 49 , shaft 10 amay be inserted through the channel defined by tapered portions 4614 ofsurgical instrument connection portion 4602, such that clamp 4618transitions from its locked state to its unlocked state to permit shaft10 a to pass into opening 4616, and back to its locked state when shaft10 a is completely disposed within opening 4616.

As the user inserts shaft 10 a within opening 4616, shaft 10 a applies adownward force against friction pad 4632, thereby causing holder 4630 tomove downward within channel 4603 and increasing the size of opening4616 until shaft 10 a is completely disposed within opening 4616 andclamp 4618 is permitted to transition back to its locked state, suchthat the shaft 10 a is positioned between the lower surface of lockingportion 4620 and friction pad 4632. Upon release of surgical instrument10 by the user, friction pad 4632 applies an upward force against shaft10 a due to the magnetic force of repulsion magnet 4512 applied againstmagnet 4640, such that shaft 10 a is pinned between the lower surface oflocking portion 4620 and friction pad 4632.

Accordingly, the magnetic field induced in ferrous rod 4510 by magnet4640 when magnet 4640 is in the position within channel 4603 responsiveto the magnetic force of repulsion magnet 4512 when coupler body 4600 iscoupled to coupler interface 4500 as well as the force applied to holder4630, and accordingly magnet 4640, by shaft 10 a via friction pad 4632,may be detected by Hall effect sensors 4514, and which may be indicativeof coupler body 4600 being coupled to coupler interface 4500, andsurgical instrument 10 being coupled to coupler body 4600. Accordingly,the system may determine that coupler body 4600 is coupled to couplerinterface 4500 and that surgical instrument 10 is coupled to couplerbody 4600, based on the strength of the magnetic field induced inferrous rod 4510 by magnet 4640.

Moreover, the position of magnet 4640 within channel 4603 will depend onthe diameter size of the surgical instrument disposed within opening4616 when coupler body 4600 is coupled to coupler interface 4500, suchthat the magnetic field induced in ferrous rod 4510 will vary based onthe surgical instrument shaft size disposed within opening 4616.Accordingly, the system may identify the precise size of the surgicalinstrument shaft based on the strength of the magnetic field induced inferrous rod 4610 by magnet 4640, as detected by Hall effect sensors4514. Based on the identified type of surgical instrument coupled tocoupler body 4600, the system may load the calibration file associatedwith the identified surgical instrument as described above.

FIGS. 50A and 50B illustrate further securement of the surgicalinstrument to surgical instrument connection portion 4602 of couplerbody 4600. FIG. 50A illustrates switch 4624 in its unlocked state whenshaft 10 a of surgical instrument 10 is positioned within opening 4616.As shown in FIG. 50A, in its unlocked state, surgical instrumentengagement portion 4626 of switch 4624 is not engaged with the outersurface of shaft 10 a. When switch 4624 is in its unlocked state,surgical instrument 10 may be moved longitudinally relative to opening4616 along shaft 10 a, e.g., to readjust the position of surgical 10relative to coupler body 4600. To secure shaft 10 a within opening 4616,such that surgical instrument 10 cannot move longitudinally relative tocoupler body 4600, the user may actuate switch 4624, e.g., via handleportion 4628, to rotate switch 4624 relative to coupler body 4600, andtransition switch 4624 to its locked state where surgical instrumentengagement portion 4626 engages with the outer surface of shaft 10 a, asshown in FIG. 50B.

As described above, the friction force applied to shaft 10 a by surgicalinstrument engagement portion 4626 facilitates securement of shaft 10 awithin coupler body 4600, such that longitudinal movement of surgicalinstrument 10 is prevented unless the longitudinal force applied tosurgical instrument 10 exceeds at least the friction force applied toshaft 10 a by surgical instrument engagement portion 4626. Moreover, thefriction force applied to shaft 10 a by surgical instrument engagementportion 4626 is such that the rotational force required to overcome thefriction force and cause rotational of shaft 10 a within opening 4616 isminimized.

Referring now to FIG. 51 , another exemplary surgical instrumentcoupling mechanism is provided. Specifically, the surgical instrumentcoupling mechanism includes coupler interface 4500, as described above,and coupler body 5100 configured to be removably coupled to a surgicalinstrument and to coupler interface 4500. Coupler body 5100 may beconstructed similar to coupler body 4600. For example, surgicalinstrument connection portion 5102, channel 5103, coupler interfaceconnection portion 5104, groove 5105, locking arms 5106, axis 5112,tapered portions 5114, opening 5116, clamp 5118, switch 5124, and holder5130 of coupler body 5100 correspond with surgical instrument connectionportion 4602, channel 4603, coupler interface connection portion 4604,groove 4605, locking arms 4606, axis 4612, tapered portions 4614,opening 4616, clamp 4618, switch 4624, and holder 4630 of coupler body4600, respectively. Coupler body 5100 differs from coupler body 4600 inthat coupler body 5100 may be configured to be removably coupled to asmaller diameter surgical instrument, e.g., a 5 mm surgical instrumentsuch as surgical instrument 12 described above with regarding to FIGS.6A to 6D.

Accordingly, as shown in FIG. 51 , opening 5116, defined at leastpartially by the lower surface of locking portion 5120 of claim 5118when clamp 5118 is in its locked state and the upper surface of frictionpad 5132 of holder 5130, may be sized and shaped to receive shaft 12 aof surgical instrument 12 therein. Upon actuation of switch 5124 to itslocked, such that surgical instrument engagement portion 5126 of switch5124 engages shaft 12 a within opening 5116, longitudinal movement ofsurgical instrument 12 is prevented unless the longitudinal forceapplied to surgical instrument 12 exceeds at least the friction forcesapplied to shaft 12 a by surgical instrument engagement portion 5126 andfriction pad 5132. Moreover, the friction forces applied to shaft 12 aby surgical instrument engagement portion 5126 and friction pad 5132 aresuch that the rotational force required to overcome the friction forcesand cause rotational of shaft 12 a within opening 5116 is minimized.Friction pad 5132 and surgical instrument engagement portion 5126 mayhave curved profiles that correspond with the outer surface of shaft 12a.

As shown in FIG. 51 , when shaft 12 a is positioned within opening 5116and clamp 5118 is in its locked state, shaft 12 a applies a downwardforce against friction pad 5132, thereby causing displacement of magnet5140 coupled to magnet harness 5138 of holder 5130, which is coupled tofriction pad 5132 via cradles 5136 of holder 5130, within channel 5103of coupler body 5100. Moreover, repulsion magnet 4512 applies a magneticforce against magnet 5140, which causes friction pad 5132 to apply anupward force against shaft 12 a, thereby maintaining the position ofmagnet 5140 within channel 5103. Accordingly, the magnetic field inducedin ferrous rod 4510 by magnet 5140 when magnet 5140 is in the positionwithin channel 5103 responsive to the magnetic force of repulsion magnet4512 when coupler body 5100 is coupled to coupler interface 4500 as wellas the force applied to holder 5130, and accordingly magnet 5140, byshaft 12 a via friction pad 5132, may be detected by Hall effect sensors4514, and which may be indicative of coupler body 5100 being coupled tocoupler interface 4500, and surgical instrument 12 being coupled tocoupler body 5100. Accordingly, the system may determine that couplerbody 5100 is coupled to coupler interface 4500 and that surgicalinstrument 12 is coupled to coupler body 5100, based on the strength ofthe magnetic field induced in ferrous rod 4510 by magnet 5140.

Referring now to FIGS. 52A to 52E, an alternative exemplary surgicalinstrument coupling mechanism is provided. Coupler interface 5200 may becoupled to or otherwise integrated with link 316, and connection portion5250 may be coupled to a coupler body, e.g., coupler body 4600 orcoupler body 5100, for removably coupling the coupler body to couplerinterface 5200. As shown in FIG. 52A, coupler interface 5200 may includeprotrusion 5204 extending from flat portion 5202. Flat portion 5202 mayhave an outer diameter that coincides with the outer diameter of link316. In addition, coupler interface 5200 may include extended portion5208 extending from flat portion 5202 and configured to be insertedwithin link 316. Like protrusion 4504, protrusion 5204 may have anon-circular profile, which corresponds to the geometry of groove 5252of connection portion 5250 of the coupler body, as described in furtherdetail below. For example, as shown in FIG. 52A, protrusion 4504 mayhave a diamond-shaped profile. Accordingly, when protrusion 5202 isdisposed within groove 5252 of connection portion 5250, rotationalmovement between coupler interface 5200 and connection portion 5250 isprevented.

Moreover, protrusion 5204 may include one or more locking portions 5206disposed on the outer surface of the sidewall of protrusion 5204. Forexample, locking portions 5206 may be indentations/grooves extendingalong the outer surface of protrusion 5204, and sized and shaped toengage with locking arms 5260 of connection portion 5250, as describedin further detail below, for securing the coupler body to couplerinterface 5200, and for securing the sterile drape between connectionportion 5250 and coupler interface 5200. Preferably, protrusion 5204includes a pair of locking portions 5206. For example, as shown in FIGS.52A and 52C, the pair of locking portions 5206 may be disposed onopposing apexes of the diamond-shaped profile of protrusion 5204.Accordingly, connection portion 5250 may be securely coupled to couplerinterface 5200 in two orientations.

As shown in FIG. 52A, coupler interface 5200 may include one or moreadditional protrusions 5210, e.g., “mating dots,” disposed on flatportion 5202. For example, coupler interface 5200 may include aplurality of protrusions 5210, preferably evenly spaced apart along flatportion 5202, e.g., adjacent to the outer edge of flat portion 5202.Protrusions 5210 may have a geometry that corresponds with the geometryof one or more additional grooves 5254 of connection portion 5250, asshown in FIG. 52B. For example, protrusions 5210 may have asemi-spherical shape, and grooves 5254 may have a correspondingsemi-spherical shape. As shown in FIG. 52B, grooves 5254 may be disposedalong connection portion 5250, such that grooves 5254 are aligned withprotrusions 5210 so that protrusions 5210 may be disposed within grooves5254 when connection portion 5250 is coupled to coupler interface 5200,as shown in FIG. 52C. Accordingly, when protrusion 5202 is disposedwithin groove 5252 of connection portion 5250, and protrusions 5210 aredisposed within grooves 5254, rotational movement between couplerinterface 5200 and connection portion 5250 is prevented. As will beunderstood by a person having ordinary skill in the art, couplerinterface 5200 and connection portion 5250 may include more or lessprotrusions 5210 and grooves 5254, respectively, that are shown in FIGS.52A and 52B. In addition, other coupler interfaces and coupler bodiesdescribed herein, e.g., coupler interface 4500 and coupler body 4600,5100, may include similar additional protrusions and grooves forproviding additional stabilization when the coupler interface is coupledto the coupler body.

As shown in FIG. 52D, connection portion 5250 may include a pair oflocking arms 5260, which may be constructed similar to locking arms 4606of connection portion 4604, for releasably securing connection portion5250 to coupler interface 5200. For example, locking arms 5260 mayinclude handle portion 5264 sized and shaped to be actuated by theuser’s fingers, and connection portion 5262 sized and shaped to engagewith locking portions 5206 of protrusion 5204. Accordingly, locking arms4606 may transition between an unlocked state where locking arms 5260are disengaged from protrusion 5204, as shown in FIG. 52D, and a lockedstate where connection portion 5262 of locking arms 5260 are engagedwith locking portions 5206 of protrusion 5204, as shown in FIG. 52E,such that locking arms 4606 are biased toward the locked state.

Referring now to FIGS. 53A and 53B, an alternative exemplary couplerbody is provided. Coupler body 5300 may be constructed similar tocoupler body 4600 and/or coupler body 5100. For example, coupler body5300 may include switch 5302, which may be constructed similar to switch4624 of coupler body 4600 and/or switch 5124 of coupler body 5100. Asshown in FIGS. 53A and 53B, switch 5302 may include one or more surgicalinstrument engagement portions 5304, each having a geometry thatcorresponds with the outer diameter of the shaft of the surgicalinstrument to be inserted within the opening of coupler body 5300. Inaddition, switch 5302 may include handle portion 5306 sized and shapedto be actuated by the user’s fingers to transition switch 5302 betweenan unlocked state, as shown in FIG. 53A, and a locked state, as shown inFIG. 53B, where surgical instrument engagement portion 5304 engages withthe surgical instrument shaft within the opening of coupler body 5300and applies a friction force to the surgical instrument shaft. As shownin FIGS. 53A and 53B, coupler body 5300 further may include one or moreprot

Referring now to FIGS. 53A and 53B, an alternative exemplary couplerbody is provided. Coupler body 5300 may be constructed similar tocoupler body 4600 and/or coupler body 5100. For example, coupler body5300 may include switch 5302, which may be constructed similar to switch4624 of coupler body 4600 and/or switch 5124 of coupler body 5100. Asshown in FIGS. 53A and 53B, switch 5302 may include one or more surgicalinstrument engagement portions 5304, each having a geometry thatcorresponds with the outer diameter of the shaft of the surgicalinstrument to be inserted within the opening of coupler body 5300. Inaddition, switch 5302 may include handle portion 5306 sized and shapedto be actuated by the user’s fingers to transition switch 5302 betweenan unlocked state, as shown in FIG. 53A, and a locked state, as shown inFIG. 53B, where surgical instrument engagement portion 5304 engages withthe surgical instrument shaft within the opening of coupler body 5300and applies a friction force to the surgical instrument shaft.

As shown in FIGS. 53A and 53B, coupler body 5300 further may include oneor more locking nubs 5308 for securing switch 5302 in the locked state.Locking nubs 5308 may have a height sufficient to permit switch 5302 topass over locking nub 5308 when a force exceeding a predeterminedthreshold is applied to switch 5302, and to prevent switch 5302 frompassing over locking nub 5308 when an insufficient amount of force isapplied to switch 5302. In some embodiments, coupler body 5300 mayinclude an internal spring coupled to switch 5302 to bias switch 5302 ina downward direction, e.g., toward the surface on which locking nubs5308 are disposed. Accordingly, when a sufficient force is applied toswitch 5302 against locking nub 5308, locking nub 5308 applies an upwardforce against switch 5302 to thereby cause the spring to compress andpermit switch 5302 to pass over locking nub 5308. Moreover, the springforce applied to switch 5302 when switch 5302 is in its locked stateprevents switch 5302 from passing over locking nub 5308 unless asufficient amount of force is applied to switch 5302 to compress thespring beyond the height of locking nub 5308.

Referring now to FIG. 54 , another alternative exemplary coupler body isprovided. Coupler body 5400 may be constructed similar to coupler body5300 and may include switch 5402 having engagement portion 5404 andhandle portion 5406. Coupler body 5400 differs from coupler body 5300 inthat instead of locking nubs 5308, coupler body 5400 may include rampedsurface 5408. As shown in FIG. 54 , ramped surface 5408 may have awave-shaped profile, e.g., two crests disposed between three valleys.For example, when switch 5402 is in its unlocked state, switch 5402 maybe disposed within valley 5410, and when switch 5402 is in its lockedstate, switch 5402 may pass over crest 5412 and disposed within valley5414. Accordingly, crest 5412 may have a height sufficient to permitswitch 5402 to pass over crest 5412 when a force exceeding apredetermined threshold is applied to switch 5402, and to prevent switch5402 from passing over crest 5412 when an insufficient amount of forceis applied to switch 5402. In some embodiments, coupler body 5400 mayinclude an internal spring coupled to switch 5402 to bias switch 5402 ina downward direction, e.g., toward ramped surface 5408. Accordingly,when a sufficient force is applied to switch 5402 against crest 5412,crest 5412 applies an upward force against switch 5402 to thereby causethe spring to compress and permit switch 5402 to pass over crest 5412.Moreover, the spring force applied to switch 5402 when switch 5402 is inits locked state within valley 5414 prevents switch 5402 from passingover crest 5412 unless a sufficient amount of force is applied to switch5402 to compress the spring beyond the height of crest 5412.

Referring now to FIG. 56 , an alternative robot arm having a motorizedjoint about axis Q3 is provided. Motorized joint 5600 may be constructedsimilar to joint 320, such that distal shoulder link 308 may be rotatedrelative to proximal shoulder link 306 about axis Q3 at joint 5600,wherein axis Q3 may be parallel to the longitudinal axis of shoulderlink 305. Motorized joint 5600 differs from joint 320 in that at leastone motor, e.g., motor M4, which may be controlled by a processor of theco-manipulation robot platform, may be operatively coupled to joint5600, to thereby apply a torque to joint 5600 to actuate rotation ofdistal shoulder link 308 relative to proximal shoulder link 306 aboutaxis Q3. For example, as shown in FIG. 56 , motor M4 may be operativelycoupled to a motion transmission mechanism coupled to distal shoulderlink 308, e.g., worm gear 5604, via gear 5602, such that actuation ofmotor M4 causes rotation of distal shoulder link 308 relative toproximal shoulder link 306 via engagement between gear 5602 and wormgear 5604.

As shown in FIG. 56 , motor M4 may be positioned adjacent to joint 5600.Unlike the other motorized joints described herein, e.g., base joint303, shoulder joint 318, and elbow joint 322, motorized joint 5600 ispreferably not be “back-drivable,” in that the user cannot actuatemotorized joint 5600, e.g., via movement of the surgical instrumentcoupled to the robot arm when the system is in co-manipulation mode.Instead, actuation of motorized joint 5600 may be conducted via one ormore actuators, e.g., an actuator disposed adjacent to motorized joint5600 and/or an actuator displayed on GUI 110, that may be actuated toautomatically cause rotation of distal shoulder link 308 relative toproximal shoulder link 306. As described above, when forces applied atthe distal end of robot arm 300 by the user serves as an input to thesystem, e.g., when the user applies a force exceeding a predeterminedthreshold in a predefined direction, the system may automaticallyactuate motorized joint 5600 to cause rotation of distal shoulder link308 relative to proximal shoulder link 306 to facilitate movement ofrobot arm 300 in the predefined direction. For example, similar to howthe system may cause the stages of platform 100 to move robot arm 300responsive to movement of the distal end of robot arm 300 by the user,e.g., back/forth along the x-axis or up/down along the z-axis, asdescribed above, the system may cause motorized joint 5600 to rotatedistal shoulder link 308 relative to proximal shoulder link 306 to moverobot arm 300 along the y-axis responsive to movement of the distal endof robot arm 300 by the user along the y-axis. Accordingly, the systemmay stop actuation of motorized joint 5600 when the force applied by theuser to the distal end of robot arm 300 drops below a predeterminedthreshold.

Moreover, as described above, as the robot arm is moved, either manuallyor automatically during setup, based on depth data obtained from the oneor more optical scanners, the system may detect when either the stagesof platform 100 or the robot arm approaches a predetermined distancethreshold relative to an object in the operating room, e.g., thesurgical bed. Accordingly, the system may automatically reconfigure therobot arm to avoid a collision with the object, e.g., by automaticallyactuating motorized joint 5600 to rotate distal shoulder link 308relative to proximal shoulder link 306. Similarly, system mayautomatically reconfigure the robot arm to avoid a collision with anobject in the operating room by automatically actuating motorized joint5600 during a surgical procedure.

While various illustrative embodiments of the invention are describedabove, it will be apparent to one skilled in the art that variouschanges and modifications may be made therein without departing from theinvention. The appended claims are intended to cover all such changesand modifications that fall within the true scope of the invention.

What is claimed:
 1. A co-manipulation surgical system to assist withlaparoscopic surgery performed using a surgical instrument having ahandle, an operating end, and an elongated shaft therebetween, theco-manipulation surgical system comprising: a robot arm comprising aproximal end, a distal end configured to be removably coupled to thesurgical instrument, a plurality of links, and a plurality of jointsbetween the proximal end and the distal end, the distal end of the robotarm comprising a coupler interface; and a coupler body configured to beremovably coupled to the coupler interface, the coupler body comprisinga lumen sized and shaped to receive the elongated shaft of the surgicalinstrument therein, the coupler body configured to transition between anopen state where the elongated shaft is slidably moveable within thelumen, and a closed state where longitudinal movement of the elongatedshaft relative to the coupler body is inhibited while rotationalmovement of the elongated shaft relative to the coupler body ispermitted responsive to movement at the handle of the surgicalinstrument, wherein, when the coupler body is coupled to the couplerinterface in the closed state, the robot arm is permitted to be freelymoveable responsive to movement at the handle of the surgical instrumentfor performing laparoscopic surgery.
 2. The co-manipulation surgicalsystem of claim 1, wherein the coupler body is disposable after a singlelaparoscopic surgery.
 3. The co-manipulation surgical system of claim 1,further comprising: a switch configured to transition between anunlocked position and a locked position, the switch comprising anengagement portion configured to engage with the elongated shaft whenthe elongated shaft is disposed within the lumen and the switch is inthe locked position to thereby secure the elongated shaft within thelumen, wherein, when the coupler body is coupled to the couplerinterface, the elongated shaft is disposed within the lumen, and theswitch is in the locked position, the robot arm is configured to befreely moveable responsive to movement at the handle of the surgicalinstrument.
 4. The co-manipulation surgical system of claim 3, wherein,when the elongated shaft is disposed within the lumen and the switch isin the locked position, the engagement portion is configured to apply afriction force against the elongated shaft, the friction forceconfigured to permit rotational movement of the elongated shaft relativeto the coupler body, while inhibiting longitudinal movement of theelongated shaft relative to the coupler body.
 5. The co-manipulationsurgical system of claim 3, wherein the switch comprises a handleportion configured to be actuated to transition the switch between theunlocked position and the locked position.
 6. The co-manipulationsurgical system of claim 3, wherein the coupler body further comprises aholder slidably disposed within the coupler body, the holder comprisinga contact surface configured to define at least a portion of the lumen,and wherein the holder is configured to be biased in a direction towardthe lumen such that, when the elongated shaft is disposed within thelumen, the contact surface is configured to engage with the elongatedshaft.
 7. The co-manipulation surgical system of claim 6, wherein thecoupler interface comprises a repulsion magnet, and wherein the holdercomprises a magnet, such that the repulsion magnet is configured toapply a magnetic force to the magnet to thereby cause the holder to bebiased in the direction toward the lumen.
 8. The co-manipulationsurgical system of claim 7, wherein the holder comprises a harnessconfigured to be coupled to the magnet, the harness sized and shaped tobe slidably disposed within a channel of the coupler body.
 9. Theco-manipulation surgical system of claim 6, wherein the contact surfaceis configured to apply a friction force against the elongated shaft whenthe elongated shaft is disposed within the lumen and the switch is inthe locked position, and wherein the friction force is configured topermit rotational movement of the elongated shaft relative to thecoupler body, while inhibiting translational movement of the elongatedshaft relative to the coupler body.
 10. The co-manipulation surgicalsystem of claim 6, further comprising: a clamp pivotally coupled to thecoupler body via a rod, the clamp configured to transition between anunlocked state where the lumen is permitted to receive the elongatedshaft and a locked state where the clamp secures the elongated shaftwithin the lumen, wherein the holder comprises one or more cradlescoupled to the contact surface, each of the one or more cradlescomprising a channel sized and shaped to slidably receive the rodtherethrough, such that the holder is configured to be slidably disposedwithin the coupler body along the rod.
 11. The co-manipulation surgicalsystem of claim 3, wherein the protrusion of the coupler interface has afirst geometry and the groove of the coupler body has a second geometrycorresponding to the first geometry, such that, when the protrusion isreceived by the groove, rotational movement between the coupler body andthe coupler interface is prohibited.
 12. The co-manipulation surgicalsystem of claim 3, wherein the coupler interface comprises one or moreadditional protrusions having a first geometry, and wherein the couplerbody comprises one or more additional grooves having a second geometry,such that, when the one or more additional protrusions are received bythe one or more additional grooves, rotational movement between thecoupler body and the coupler interface is prohibited.
 13. Theco-manipulation surgical system of claim 1, wherein the couplerinterface comprises a protrusion, and wherein the coupler body comprisesa groove configured to receive the protrusion of the coupler interface.14. The co-manipulation surgical system of claim 13, wherein theprotrusion comprises one or more indentations, and wherein the couplerbody comprises: one or more locking arms configured to transitionbetween a locked configuration where at least a portion of the one ormore locking arms extend within the groove of the coupler body, and anunlocked configuration where the one or more locking arms do not extendwithin the groove of the coupler body, wherein the protrusion of thecoupler interface is configured to be received by the groove of thecoupler body when the one or more locking arms are in the unlockedconfiguration, and wherein the at least a portion of the one or morelocking arms extend within the one or more indentations of theprotrusion when the protrusion is disposed within the groove and thelocking arms are in the locked configuration to thereby secure thecoupler body to the coupler interface.
 15. The co-manipulation surgicalsystem of claim 14, wherein the one or more locking arms are biasedtoward the locked configuration.
 16. The co-manipulation surgical systemof claim 14, wherein each of the one or more locking arms comprises ahandle portion configured to be actuated to transition the one or morelocking arms from the locked configuration to the unlockedconfiguration.
 17. The co-manipulation surgical system of claim 1,wherein the coupler body and the coupler interface are configured toreceive a sterile drape therebetween, such that the sterile drapeprevents contact between the surgical instrument and the robot armduring the laparoscopic surgery.
 18. The co-manipulation surgical systemof claim 1, wherein the coupler body comprises one or more taperedsurfaces configured to guide the elongated shaft into the lumen andfacilitate self-alignment of the distal end of the robot arm relative tothe surgical instrument by causing the coupler body and the couplerinterface to rotate to align the lumen with the elongated shaft as theelongated shaft is inserted along the one or more tapered surfaces intothe lumen.
 19. The co-manipulation surgical system of claim 1, whereinthe coupler body comprises a clamp configured to transition between anunlocked state where the lumen is permitted to receive the elongatedshaft and a locked state where the clamp secures the elongated shaftwithin the lumen.
 20. The co-manipulation surgical system of claim 19,wherein the clamp is configured to be biased toward the locked state.21. The co-manipulation surgical system of claim 20, wherein at least aportion of the clamp comprises a tapered surface configured to guide theelongated shaft into the lumen and to facilitate transitioning of theclamp from the locked state to the unlocked state responsive to a forceapplied to the tapered surface by the elongated shaft as the elongatedshaft is inserted into the lumen.
 22. A method for assisting withlaparoscopic surgery using a robot arm configured to be removablycoupled to a surgical instrument having a handle, an operating end, andan elongated shaft therebetween, the method comprising: removablycoupling a coupler body to a coupler interface at a distal end of therobot arm; inserting the elongated shaft of the surgical instrument intoa lumen of the coupler body; transitioning the coupler body from an openstate where the elongated shaft is slidably moveable within the lumen toa closed state where longitudinal movement of the elongated shaftrelative to the coupler body is inhibited while rotational movement ofthe elongated shaft relative to the coupler body is permitted responsiveto movement at the handle of the surgical instrument; and freely movingthe robot arm by moving the handle of the surgical instrument when thecoupler body is coupled to the coupler interface in the closed state toperform the laparoscopic surgery.
 23. The method of claim 22, whereinremovably coupling the coupler body to the coupler interface comprises:actuating one or more locking arms of the coupler body to transition theone or more locking arms from a locked configuration where at least aportion of the one or more locking arms extend within a groove of thecoupler body, to an unlocked configuration where the one or more lockingarms do not extend within the groove; inserting a protrusion of thecoupler interface within a groove of the coupler body; and releasing theone or more locking arms to transition the one or more locking arms fromthe unlocked configuration to the locked configuration, such that the atleast a portion of the one or more locking arms extend within one ormore indentations of the protrusion to thereby secure the coupler bodyto the coupler interface.
 24. The method of claim 22, wherein insertingthe elongated shaft of the surgical instrument within the lumen of thecoupler body comprises guiding the elongated shaft along one or moretapered surfaces of the coupler body into the lumen.
 25. The method ofclaim 24, wherein guiding the elongated shaft along one or more taperedsurfaces of the coupler body into the lumen comprises rotating thecoupler body and the coupler interface to facilitate self-alignment ofthe lumen with the elongated shaft as the elongated shaft is insertedalong the one or more tapered surfaces into the lumen.
 26. The method ofclaim 22, wherein inserting the elongated shaft of the surgicalinstrument within the lumen of the coupler body comprises: actuating aclamp of the coupler body to transition the clamp from a locked state toan unlocked state where the lumen is permitted to receive the elongatedshaft; inserting the elongated shaft of the surgical instrument withinthe lumen; and releasing the clamp to transition the clamp from theunlocked state to the locked state, such that the clamp secures theelongated shaft within the lumen.
 27. The method of claim 22, whereintransitioning the coupler body from the open state to the closed statecomprises transitioning a switch of the coupler body from an unlockedposition where the elongated shaft is slidably moveable within thelumen, to a locked position where an engagement portion of the switchengages with the elongated shaft disposed within the lumen to therebyinhibit longitudinal movement of the elongated shaft relative to thecoupler body while permitting rotational movement of the elongated shaftrelative to the coupler body.
 28. The method of claim 27, wherein, whenthe elongated shaft is disposed within the lumen and the switch is inthe locked position, the engagement portion applies a friction forceagainst the elongated shaft, the friction force configured to permitrotational movement of the elongated shaft relative to the coupler body,while inhibiting longitudinal movement of the elongated shaft relativeto the coupler body.
 29. The method of claim 22, wherein the couplerinterface comprises a repulsion magnet, and wherein the coupler bodycomprises a holder slidably disposed within the coupler body, the holdercomprising a magnet and a contact surface configured to define at leasta portion of the lumen, such that when the coupler body is removablycoupled to the coupler interface, the repulsion magnet applies amagnetic force to the magnet to bias the holder in a direction towardthe lumen.
 30. The method of claim 22, further comprising positioning asterile drape between the coupler body and the coupler interface priorto removably coupling the coupler body to the coupler interface.